Information processing apparatus and information processing method

ABSTRACT

There is provided an information processing apparatus including: a processing unit configured to perform a summarization process of summarizing content of speech indicated by voice information based on speech of a user on a basis of acquired information indicating a weight related to a summary.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2016/080485 filed on Oct. 14, 2016, which claimspriority benefit of Japanese Patent Application No. JP 2016-011224 filedin the Japan Patent Office on Jan. 25, 2016. Each of theabove-referenced applications is hereby incorporated herein by referencein its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, and a program.

BACKGROUND ART

A technique of summarizing an electronic document has been developed. Asa technique for summarizing an electronic document and adding a tagindicating copyright information to a generated summary, for example,there is a technique disclosed in Patent Literature 1.

CITATION LIST Patent Literature

Patent Literature 1: JP 2001-167114A

DISCLOSURE OF INVENTION Technical Problem

In a case in which a person speaking (hereinafter referred to as a“speaker”) speaks, it is difficult for the speaker to speak only contentwhich the speaker desires to convey.

For this reason, if a case in which communication is performed by speechis assumed, for example, often, “content other than the content whichthe speaker desires to convey (that is, unnecessary content) is conveyedto a partner performing communication in addition to the content whichthe speaker desires to convey.” Therefore, in a case in whichcommunication is performed by speech, it may “take time for a partnerperforming communication to understand content which the speaker desiresto convey.”

Further, if a case of translating content of speech into anotherlanguage is assumed, for example, since “the speaker speakers contentother than the content which the speaker desires to convey in additionto the content which the speaker desires to convey,” for example, it may“take time to translate” or “a translation result which is not intendedby the speaker” may be obtained.

Here, as a method of reducing a possibility of the occurrence of “anevent caused by the speaker's difficulty speaking only the content whichthe speaker desires to convey” such as “it taking time for the partnerperforming the communication to understand content which the speakerdesires to convey” or “it taking time to translate,” there is a methodto further simplify content of speech of the speaker.

The present disclosure proposes an information processing apparatus, aninformation processing method, and a program which are novel andimproved and capable of summarizing content of speech.

Solution to Problem

According to the present disclosure, there is provided an informationprocessing apparatus including: a processing unit configured to performa summarization process of summarizing content of speech indicated byvoice information based on speech of a user on a basis of acquiredinformation indicating a weight related to a summary.

In addition, according to the present disclosure, there is provided aninformation processing method that is executed by an informationprocessing apparatus, the information processing method including: astep of performing a summarization process of summarizing content ofspeech indicated by voice information based on speech of a user on abasis of acquired information indicating a weight related to a summary.

In addition, according to the present disclosure, there is provided aprogram for causing a computer to implement: a function of performing asummarization process of summarizing content of speech indicated byvoice information based on speech of a user on a basis of acquiredinformation indicating a weight related to a summary.

Advantageous Effects of Invention

According to the present disclosure, it is possible to summarize contentof speech.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram for describing an example of a use caseto which an information processing method of the present embodiment isapplied.

FIG. 2 is an explanatory diagram for describing an example of a use caseto which an information processing method of the present embodiment isapplied.

FIGS. 3A, 3B, and 3C are explanatory diagrams for describing an exampleof a use case to which an information processing method of the presentembodiment is applied.

FIGS. 4A and 4B are explanatory diagrams for describing an example of ause case to which an information processing method of the presentembodiment is applied.

FIG. 5 is an explanatory diagram for describing an example of a use caseto which an information processing method of the present embodiment isapplied.

FIG. 6 is an explanatory diagram illustrating an example of a table forsetting a weight related to a summary according to the presentembodiment.

FIG. 7 is an explanatory diagram illustrating an example of a table forsetting a weight related to a summary according to the presentembodiment.

FIG. 8 is an explanatory diagram illustrating an example of a table forsetting a weight related to a summary according to the presentembodiment.

FIGS. 9A and 9B are explanatory diagrams for describing an example of asummarization process according to a first information processingmethod.

FIGS. 9C and 9D are explanatory diagrams for describing an example of asummarization process according to a first information processingmethod.

FIGS. 9E and 9F are explanatory diagrams for describing an example of asummarization process according to a first information processingmethod.

FIG. 10 is an explanatory diagram illustrating an example of anotification by a visual method implemented by a notification controlprocess according to a second information processing method.

FIG. 11 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIG. 12 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIG. 13 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIG. 14 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIG. 15 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIG. 16A is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIGS. 16B and 16C are explanatory diagrams for describing an example ofa notification control process according to the second informationprocessing method.

FIGS. 16D and 16E are explanatory diagrams for describing an example ofa notification control process according to the second informationprocessing method.

FIG. 17 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIG. 18 is an explanatory diagram for describing an example of anotification control process according to the second informationprocessing method.

FIGS. 19A and 19B are explanatory diagrams for describing an example ofa notification control process according to the second informationprocessing method.

FIGS. 20A and 20B are explanatory diagrams for describing an example ofa notification control process according to the second informationprocessing method.

FIGS. 21A, 21B, and 21C are explanatory diagrams for describing anexample of a notification control process according to the secondinformation processing method.

FIG. 22 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 23 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 24 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 25 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 26 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 27 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 28 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 29 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 30 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 31 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 32 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 33 is a flowchart illustrating an example of a process according toan information processing method of the present embodiment.

FIG. 34 is a block diagram illustrating an example of a configuration ofan information processing apparatus according to the embodiment.

FIG. 35 is an explanatory diagram illustrating an example of a hardwareconfiguration of the information processing apparatus according to theembodiment.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. Notethat, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

Further, the following description will proceed in the order describedbelow.

1. Information processing method according to embodiment

2. Information processing apparatus according to embodiment

3. Program according to present embodiment

(Information Processing Method According to Present Embodiment)

First, an information processing method of the present embodiment willbe described. A case in which a process according to an informationprocessing method of the present embodiment is performed by aninformation processing apparatus according to the present embodimentwill be described below as an example.

Further, the following description will proceed with an example in whichthe information processing method of the present embodiment is dividedinto a first information processing method and a second informationprocessing method. Further, the following description will proceedmainly with a case in which the same information processing apparatusperforms both a process according to the first information processingmethod and a process according to the second information processingmethod, but an information processing apparatus performing the processaccording to the first information processing method may be differentfrom an information processing apparatus performing the processaccording to the second information processing method.

Further, in the following description, a person who is a target of theprocess according to the information processing method of the presentembodiment is referred to as a “user.” Examples of the user according tothe present embodiment include a “speaker (or a person who can be aspeaker)” (in a case in which the first information processing method tobe described later is performed) or a “manipulator of a manipulationdevice related to a notification” (in a case in which the secondinformation processing method to be described later is performed).

[1] Overview of Information Processing Method According to PresentEmbodiment

[1-1] Overview of First Information Processing Method

As mentioned above, as a method to further reduce the possibility of theoccurrence of an “event caused by the speaker's difficulty speaking onlythe content which the speaker desires to convey,” a method of furthersimplifying content of speech of the speaker is considered.

In this regard, the information processing apparatus according to thepresent embodiment performs a process of summarizing content of speech(hereinafter referred to as a “summarization process”) as the processaccording to the first information processing method. The informationprocessing apparatus according to the present embodiment summarizescontent of speech indicated by voice information based on the speech ofthe user on the basis of acquired information indicating a weightrelated to a summary. Examples of the summarization according to thepresent embodiment include selecting content of speech on the basis ofthe weight related to the summary and extracting a part from content ofspeech on the basis of the weight related to the summary.

The information indicating the weight related to the summary is, forexample, data indicating the weight related to the summary stored in atable (or a database; the same applies hereinafter) for setting theweight related to the summary to be described later. Further, theinformation indicating the weight related to the summary may be dataindicating that the weight related to the summary is relatively large orsmall. For example, the information indicating the weight related to thesummary is acquired with reference to the table for setting the weightrelated to the summary to be described later, or the like.

Here, the voice information according to the present embodiment is voicedata including voice based on speech of the speaker. The voiceinformation according to the present embodiment is generated, forexample, by picking up voice based on the speech of the speaker througha voice input device such as a microphone. Further, the voiceinformation according to the present embodiment may be a digital signalobtained by converting an analog signal generated in accordance with thevoice picked up through the voice input device through ananalog-to-digital (AD) converter. Further, the voice input device (orthe voice input device and the AD converter) may be installed in theinformation processing apparatus according to the present embodiment oran external device of the information processing apparatus according tothe present embodiment.

The content of the speech indicated by the voice information is, forexample, a character string indicated by text data (hereinafter referredto as “voice text information”) obtained as a result of performing anarbitrary voice recognition process on the voice information. Theinformation processing apparatus according to the present embodimentrecognizes the character string indicated by the voice text informationas the content of the speech indicated by the voice information andsummarizes the character string indicated by the voice text information.

Here, the voice recognition process on the voice information may beperformed by the information processing apparatus according to thepresent embodiment or may be performed in an external apparatus of theinformation processing apparatus according to the present embodiment. Ina case in which the information processing apparatus according to thepresent embodiment performs the voice recognition process, theinformation processing apparatus according to the present embodimentsummarizes the character string indicated by the voice text informationobtained as a result of performing the voice recognition process on theacquired voice information. Further, in a case in which the externalapparatus of the information processing apparatus according to thepresent embodiment performs the voice recognition process, theinformation processing apparatus according to the present embodimentsummarizes the character string indicated by the voice text informationacquired from the external apparatus.

Further, in the information processing apparatus according to thepresent embodiment or the external apparatus, the voice recognitionprocess may be repeated, for example, periodically/non-periodically ormay be performed in accordance with a predetermined trigger such as atiming at which the voice information is acquired. Further, in theinformation processing apparatus according to the present embodiment orthe external apparatus, the voice recognition process may be performed,for example, when a predetermined manipulation such as a manipulation ofstarting voice recognition related to a summary is performed.

The weight related to the summary according to the present embodiment isan index for extracting a more significant word (that is, a word whichthe speaker is considered to wish to convey) from the content of thespeech indicated by the voice information. On the basis of the weightrelated to the summary according to the present embodiment, the contentof the speech indicated by the voice information is summarized, and amore significant word corresponding to the weight related to the summaryis included in the summarized content of the speech.

The weight related to the summary according to the present embodiment isset on the basis of at least one of, for example, the voice information,information related to the user, information related to an application,information related to an environment, and information related to adevice (1 or more of these) which will be described below.

Here, the information related to the user according to the presentembodiment includes at least one of, for example, state information ofthe user indicating a state of the user and manipulation information ofthe user based on a manipulation of the user.

Examples of the state of the user include a behavior performed by theuser (including an action such as a gesture), an emotional state of theuser, and the like. For example, the state of the user is estimatedaccording to an arbitrary behavior estimation process or an arbitraryemotion estimation process using one or more of biometric information ofthe user obtained from an arbitrary biosensor, a detection result of amotion sensor such as a speed sensor or an angular velocity sensor, acaptured image captured by an imaging device, and the like. The processrelated to the estimation of the state of the user may be performed bythe information processing apparatus according to the present embodimentor may be performed in the external apparatus of the informationprocessing apparatus according to the present embodiment. Further, themanipulation of the user includes, for example, various manipulationssuch as a manipulation of starting voice recognition related to asummary and a manipulation of activating a predetermined application.

Further, the information related to the application indicates, forexample, an execution state of the application.

Further, the information related to the environment indicates, forexample, a situation around the user (or a situation in which the useris placed). The information related to the environment is, for example,data indicating a level of noise around the user, and the like. Forexample, the level of the noise around the user is specified byextracting non-speech information from the voice information generatedby the microphone and performing a threshold value process using one ormore threshold values for level classification. A process related toacquisition of the information related to the environment may beperformed by the information processing apparatus according to thepresent embodiment or may be performed in the external apparatus of theinformation processing apparatus according to the present embodiment.

Further, the information related to the device indicates, for example,either or both of a type of device and a state of the device. Examplesof the state of the device include, for example, a processing load of aprocessor installed in the device.

A specific example of a process related to setting of the weight relatedto the summary will be described later.

As the summarization process according to the first informationprocessing method is performed, the content of the speech indicated bythe voice information is summarized. Therefore, it is possible tofurther simplify the content of the speech of the speaker indicated bythe voice information.

Further, in the summarization process according to the first informationprocessing method, for example, since the content of the speech issummarized on the basis of the weight related to the summary set asdescribed above, more significant words corresponding to the weightrelated to the summary are included in the summarized content of thespeech.

Therefore, when the summarization process according to the firstinformation processing method is performed, it is possible to obtain asummarization result capable of reducing the possibility of theoccurrence of “an event caused by the speaker's difficulty speaking onlythe content which the speaker desires to convey” such as “it taking timefor the partner performing the communication to understand content whichthe speaker desires to convey” or “it taking time to translate.”

[1-2] Overview of Second Information Processing Method

When the summarization process according to the first informationprocessing method is performed, it is possible to obtain the content ofthe speech indicated by the summarized voice information.

The information processing apparatus according to the present embodimentperforms a process of controlling a notification of notification contenton the basis of summary information (hereinafter referred to as a“notification control process”) as the process according to the secondinformation processing method.

Here, the summary information according to the present embodimentindicates content of summarized speech corresponding to voiceinformation based on speech of a first user. The summary information isobtained, for example, by performing the summarization process accordingto the first information processing method. Further, the summarizedcontent of the speech indicated by the summary information is notlimited to the above example and may be content of a summary obtained byan arbitrary method capable of summarizing the content of the speechindicated by the voice information based on the speech of the user. Acase in which the summary information indicates the summarized contentof the speech obtained by performing the summarization process accordingto the first information processing method will be described below as anexample.

Further, the information processing apparatus according to the presentembodiment controls a notification of notification content for a seconduser. Here, the notification content for the second user may be, forexample, summarized content of the speech indicated by the summaryinformation or may be something other than the summarized content of thespeech indicated by the summary information such as content which isdifferent from the summarized content of the speech in a notificationorder, or content obtained by translating the summarized content of thespeech. Further, the first user according to the present embodiment andthe second user according to the present embodiment may be different ormay be the same. An example of a case in which the first user and thesecond user are different is a case in which the first user is aspeaker, and the second user is a partner performing communication.Further, an example of a case in which the first user and the seconduser are the same is a case in which the first user and the second userare the same speaker.

The information processing apparatus according to the present embodimentcauses a notification of the notification content to be given, forexample, using either or both of a notification based on a visual methodand a notification based on an auditory method.

In a case in which the notification is given using the visual method,the information processing apparatus according to the present embodimentgives the notification by, for example, causing the notification contentto be displayed on a display screen of a display device. The informationprocessing apparatus according to the present embodiment causes thenotification content to be displayed on the display screen of thedisplay device by transmitting a display control signal includingdisplay data corresponding to the notification content and a displaycommand to the display device.

Here, examples of the display screen which is caused to display thenotification content include a display device constituting a displayunit (to be described later) of the information processing apparatusaccording to the present embodiment and an external display device ofthe information processing apparatus according to the presentembodiment. In a case in which the display screen which is caused todisplay the notification content is the external display device, theinformation processing apparatus according to the present embodimentcauses, for example, a communication unit (to be described later)installed in the information processing apparatus according to thepresent embodiment or an external communication device of theinformation processing apparatus according to the present embodiment totransmit the display control signal to the external display device.

Further, in a case in which the notification is given using the auditorymethod, the information processing apparatus according to the presentembodiment gives the notification by causing, for example, thenotification content to be output from a voice output device such as aspeaker by voice (which may include music). The information processingapparatus according to the present embodiment causes the notificationcontent to be output from the voice output device by voice bytransmitting a voice output control signal including voice dataindicating voice corresponding to the notification content and a voiceoutput command to the voice output device.

Here, the voice output device which is caused to output the notificationcontent by voice may be, for example, a voice output device installed inthe information processing apparatus according to the present embodimentor may be an external voice output device of the information processingapparatus according to the present embodiment. In a case in which thevoice output device which is caused to output the notification contentby voice is the external voice output device, the information processingapparatus according to the present embodiment causes, for example, acommunication unit (to be described later) installed in the informationprocessing apparatus according to the present embodiment or an externalcommunication device of the information processing apparatus accordingto the present embodiment to transmit the voice output control signal tothe external voice output device.

Further, the notification content notification method in the informationprocessing apparatus according to the present embodiment is not limitedto one or both of the notification method using the visual method andthe notification method using the auditory method. For example, theinformation processing apparatus according to the present embodiment cancause a notification of a delimiter in the notification content to begiven using a haptic notification method, for example by causing avibration device to vibrate, and the like.

For example, a notification of the notification content based on thesummarized content of the speech obtained by the summarization processaccording to the first information processing method is given byperforming a notification control process according to the secondinformation processing method.

Here, as described above, the content of the summarized speech obtainedby the summarization process according to the first informationprocessing method corresponds to the summarization result capable ofreducing the possibility of the occurrence of “an event caused by thespeaker's difficulty speaking only the content which the speaker desiresto convey.”

Therefore, when the summarization process according to the secondinformation processing method is performed, a notification of thenotification content is given, and thus it is possible to reduce thepossibility of the occurrence of “an event caused by the speaker'sdifficulty speaking only the content which the speaker desires toconvey” such as “it taking time for the partner performing thecommunication to understand content which the speaker desires to convey”or “it taking time to translate.”

[1-3] Other Processes According to Information Processing MethodAccording to Present Embodiment

Further, the process according to the information processing method ofthe present embodiment is not limited to the summarization processaccording to the first information processing method and thenotification control process according to the second informationprocessing method.

For example, the process according to the information processing methodof the present embodiment may further include a process of translatingthe content of the speech summarized according to the summarizationprocess according to the first information processing method intoanother language (hereinafter referred to as a “translation process”).When the translation process is performed, the summarized content of thespeech is translated from a first language corresponding to the voiceinformation based on the speech into a second language different fromthe first language. The translated summarized content of the speechobtained by performing the translation process is referred to as a“translation result.”

Here, the translation process according to the present embodiment may becarried out as a part of the process according to the first informationprocessing method or as a part of the process according to the secondinformation processing method.

Further, the process according to the information processing method ofthe present embodiment may further include a recording control processof causing either or both of the result of the summarization processaccording to the first information processing method and the result ofthe translation process according to the present embodiment to berecorded in an arbitrary recording medium.

Further, in the recording control process, for example, “either or bothof the result of the summarization process according to the firstinformation processing method and the result of the translation processaccording to the present embodiment,” and information related to theuser such as “position information (to be described later) correspondingto the user and biometric information of the user obtained from anarbitrary biosensor or the like” may be associated with each other andrecorded as a log. As the log is recorded in the recording medium, forexample, an application in which “the user looks back later on recordingof travel or the like” is implemented.

[2] One Example of Use Case to which Information Processing MethodAccording to Present Embodiment is Applied

Next, an example of the process according to the information processingmethod of the present embodiment will be described while describing anexample of a use case to which the information processing method of thepresent embodiment is applied. As the use case to which the informationprocessing method of the present embodiment is applied, a case in whichthe information processing method of the present embodiment is appliedto “conversation support” (including a case in which translation isperformed as will described later) will be described below.

Further, the use case to which the information processing method of thepresent embodiment is applied is not limited to the “conversationsupport.” For example, the information processing method of the presentembodiment can be applied to arbitrary use cases in which the content ofthe speech indicated by voice information is summarized as describedbelow.

-   -   “Conference transcription” implemented by summarizing content of        speech indicated by voice information indicating voice in a        conference which is generated by an integrated circuit (IC)        recorder or the like    -   “Automatic program telop generation” implemented by summarizing        content of speech indicated by voice information indicating        voice in a television program    -   Either or both of “automatic conference telop generation” and        “conference transcription” implemented by summarizing content of        speech indicated by voice information indicating voice in a        television conference

FIG. 1 to FIG. 5 are explanatory diagrams for describing an example of ause case to which the information processing method of the presentembodiment is applied.

A person indicated by “U1” in FIGS. 1, 2, and 5 corresponds to the useraccording to the present embodiment. Further, a person indicated by “U2”in FIG. 2 and FIG. 5 corresponds to a partner who performs communicationwith the user U1. Hereinafter, the person indicated by “U1” in FIGS. 1,2, and 5 is referred to as a “user U1,” and the person indicated by “U2”in FIGS. 2 and 5 is referred to as a “communication partner U2.”Further, in the following description, a native language of thecommunication partner U2 is assumed to be Japanese.

In FIGS. 1, 2, and 5, an example in which the user U1 is wearing aneyewear type apparatus including a display screen is illustrated.Further, a voice input device such as a microphone, a voice outputdevice such as a speaker, and an imaging device are connected to theeyewear type apparatus worn by the user U1 illustrated in FIGS. 1, 2,and 5.

Further, in an example of the use case described below, examples of theinformation processing apparatus according to the present embodimentinclude a wearable apparatus used in a state in which it is worn on thebody of the user U1 such as the eyewear type apparatus illustrated inFIG. 1, a communication apparatus such as a smartphone, a computer suchas a server, and the like. Further, the information processing apparatusaccording to the present embodiment is not limited to the exampledescribed above. An application example of the information processingapparatus according to the present embodiment will be described later.

An example of the use case to which the information processing method ofthe present embodiment is applied will be described below with referenceto FIGS. 1, 2, 3A, 3B, 3C, 4A, 4B, and 5.

A case in which the user U1 speaking English arrives at an airport inJapan by airplane is assumed.

(a) Example of Process Related to Setting of Weight Related to Summary

The information processing apparatus according to the presentembodiment, for example, sets a weight related to a summary using atable for setting the weight related to the summary. Here, the table forsetting the weight related to the summary may be stored in a storageunit (to be described later) of the information processing apparatusaccording to the present embodiment or may be stored in an externalrecording medium of the information processing apparatus according tothe present embodiment. The information processing apparatus accordingto the present embodiment uses, for example, the table for setting theweight related to the summary with reference to the storage unit (to bedescribed later) or the external recording medium as appropriate.

Further, for example, the information processing apparatus according tothe present embodiment can decide the weight related to the summaryaccording to an arbitrary algorithm of deciding the weight related tothe summary and set the weight related to the summary.

FIGS. 6 to 8 are explanatory diagrams illustrating an example of thetable for setting the weight related to the summary according to thepresent embodiment.

FIG. 6 illustrates an example of a table for specifying the weightrelated to the summary and illustrates an example of a table in which aweight is assigned to each type of weight related to a summary for eachword of a registered vocabulary. Here, in FIG. 6, a combinationindicated by a value “1” among combinations of types of weight relatedto the summary and vocabulary words corresponds to a weightedcombination. Further, in FIG. 6, a combination indicated with a value“0” among combinations of types of weight related to the summary andvocabulary words corresponds to a non-weighted combination.

Further, FIGS. 7 and 8 illustrate examples of tables for specifyingtypes of the weight related to the summary. FIG. 7 illustrates anexample of a table in which schedule content specified from a state of ascheduling application (or schedule content estimated from the state ofthe scheduling application) is associated with a type of weight relatedto the summary. Further, FIG. 8 illustrates an example of a table inwhich a behavior of the user (an example of the state of the user) isassociated with a type of weight related to the summary.

For example, the information processing apparatus according to thepresent embodiment sets the weight related to the summary using both thetables for specifying the type of weight related to the summary asillustrated in FIGS. 7 and 8 and the table for specifying the weightrelated to the summary as illustrated in FIG. 6 as the table for settingthe weight related to the summary.

Further, the example of the table for specifying the type of weightrelated to the summary according to the present embodiment is notlimited to the examples illustrated in FIGS. 7 and 8, and the example ofthe table for specifying the weight related to the summary is notlimited to the example illustrated in FIG. 6. Further, the table forsetting the weight related to the summary according to the presentembodiment may be provided in each language such as Japanese, English,Chinese, or the like.

Further, the information processing apparatus according to the presentembodiment can set the weight related to the summary using only thetable for specifying the weight related to the summary as illustrated inFIG. 6 in a case in which deciding the type of weight related to thesummary on the basis of at least one of, for example, the voiceinformation, the information related to the user, the informationrelated to the application, the information related to the environment,and the information related to the device.

The information processing apparatus according to the present embodimentdecides the type of weight related to the summary by selecting the typeof weight related to the summary associated with a recognition resultfrom the table for specifying the weight related to the summaryillustrated in FIG. 6 on the basis of the recognition result based on atleast one of, for example, the voice information, the informationrelated to the user, the information related to the application, theinformation related to the environment, and the information related tothe device. Then, for example, the information processing apparatusaccording to the present embodiment sets a weighting to a vocabularyword corresponding to the combination indicated by the value “1” amongthe combinations of decided types of weight related to the summary andvocabulary words with reference to the table for specifying the weightrelated to the summary as illustrated in FIG. 6.

As a specific example, the information processing apparatus according tothe present embodiment sets the weight related to the summary, forexample, by performing any of the following processes (a-1) to (a-5).

Further, an example related to the setting of the weight related to thesummary is not limited to the examples described in the followingprocesses (a-1) to (a-5). For example, the information processingapparatus according to the present embodiment can set the weight relatedto the summary in accordance with a language recognized on the basis ofthe voice information as well. As an example of the setting of theweight related to the summary according to the language, there are, forexample, “increasing a weight of a verb in a case in which the languagerecognized on the basis of the voice information is Japanese,”“increasing a weight of a noun if the language recognized on the basisof the voice information is English,” and the like. Further, forexample, the information processing apparatus according to the presentembodiment may set each of the weight related to the summary accordingto the situation around the user indicated by the information related tothe environment and the weight related to the summary according tocontent indicated by the information related to the device (for example,the type of device or the like).

(a-1) First Example of Setting of Weight Related to Summary: One Exampleof Setting of Weight Related to Summary Based on State of User Indicatedby State Information of User Included in Information Related to User

For example, if the user U1 manipulates an apparatus such as asmartphone to activate the scheduling application and checks adestination, the information processing apparatus according to thepresent embodiment recognizes that the user U1 is moving toward thedestination. Then, the information processing apparatus according to thepresent embodiment sets the weight related to the summary correspondingto the recognition result with reference to the table for setting theweight related to the summary.

As a specific example, the information processing apparatus according tothe present embodiment specifies “time” corresponding a behavior“moving” as the type of weight related to the summary from the table forspecifying the type of weight related to the summary illustrated in FIG.8 on the basis of the recognition result obtained as described aboveindicating that the user U1 is moving toward the destination. Further,the information processing apparatus according to the present embodimentsets a weighting to a vocabulary word corresponding to the combinationindicated by the value “1” among the combinations of specified types ofweight related to the summary and vocabulary words with reference to thetable for specifying the weight related to the summary illustrated inFIG. 6. In a case in which the table for specifying the weight relatedto the summary illustrated in FIG. 6 is used, the weighting is set tothe vocabulary “morning,” “when,” . . . .

Further, in a case in which the user U1 manipulates an apparatus such asa smartphone and activates the game application, the informationprocessing apparatus according to the present embodiment recognizes thatthe user U1 is playing the game. Then, the information processingapparatus according to the present embodiment sets the weight related tothe summary corresponding to the recognition result with reference tothe table for setting the weight related to the summary.

For example, the information processing apparatus according to thepresent embodiment specifies “game terms” corresponding to a behavior“playing a game” as the type of weight related to the summary from thetable for specifying the type of weight related to the summaryillustrated in FIG. 8 on the basis of the recognition result obtained asdescribed above indicating that the user U1 is playing a game. Further,the information processing apparatus according to the present embodimentsets a weighting to a vocabulary word corresponding to the combinationindicated by the value “1” among the combinations of decided types ofweight related to the summary and vocabulary words with reference to thetable for specifying the weight related to the summary illustrated inFIG. 6.

Further, the information processing apparatus according to the presentembodiment can decide the type of weight related to the summaryassociated with the recognition result such as “game terms” included inthe table for specifying the weight related to the summary illustratedin FIG. 6 as the type of weight related to the summary on the basis ofthe recognition result obtained as described above indicating that theuser U1 is playing a game. Further, the information processing apparatusaccording to the present embodiment sets a weighting to a vocabularyword corresponding to the combination indicated by the value “1” amongthe combinations of decided types of weight related to the summary andvocabulary words with reference to the table for specifying the weightrelated to the summary illustrated in FIG. 6.

Further, for example, the information processing apparatus according tothe present embodiment can set the weight related to the summary on thebasis of the recognition result of the state of the user U1 estimated onthe basis of the detection result of the motion sensor such as theacceleration sensor or the angular velocity sensor installed in anapparatus such as a smartphone used by the user U1.

For example, in a case in which a recognition result indicating that theuser U1 is eating is obtained on the basis of the detection result ofthe motion sensor, “dish” corresponding to a behavior “eating” isspecified as the type of weight related to the summary from the tablefor specifying the type of weight related to the summary illustrated inFIG. 8. Further, the information processing apparatus according to thepresent embodiment sets a weighting to a vocabulary word correspondingto the combination indicated by the value “1” among the combinations ofdecided types of weight related to the summary and vocabulary words withreference to the table for specifying the weight related to the summaryillustrated in FIG. 6.

(a-2) Second Example of Setting of Weight Related to Summary: OneExample of Setting of Weight Related to Summary Based on VoiceInformation

The information processing apparatus according to the present embodimentsets the weight related to the summary on the basis of the voiceinformation.

The information processing apparatus according to the present embodimentdecides the type of weight related to the summary on the basis of, forexample, the voice information as described below.

-   -   In a case in which an average frequency band of the voice        indicated by the voice information is, for example, 300 to 550        [Hz], “male” is decided as the type of weight related to the        summary.    -   In a case in which the average frequency band of the voice        indicated by the voice information is, for example, 400 to 700        [Hz], “female” is decided as the type of weight related to the        summary.    -   In a case in which sound pressure and volume of the voice        indicated by the voice information are equal to or larger than a        set first threshold value or in a case in which the sound        pressure and volume of the voice indicated by the voice        information are larger than the first threshold value, either or        both of “anger” and “joy” are decided as the type of weight        related to the summary.    -   In a case in which the sound pressure and volume of the voice        indicated by the voice information are equal to or less than a        set second threshold value or in a case in which the sound        pressure and volume of the voice indicated by the voice        information are less than the second threshold value: one or        more of “sadness,” “discomfort,” “distress,” and “anxiety” are        decided.    -   In a case in which a pitch (a sound pitch) or a speaking speed        (amount of phoneme per unit time) of the voice indicated by the        voice information is larger than a set third threshold value or        in a case in which the pitch or the speaking speed of the voice        indicated by voice information is equal to or larger than the        third threshold value, “excitement” is decided as the type of        weight related to the summary. In a case in which the pitch or        the speaking speed of the voice indicated by voice information        is less than a set fourth threshold value or in a case in which        the pitch or the speaking speed of the voice indicated by voice        information is equal to or less than the fourth threshold value,        “balance” is decided as the type of weight related to the        summary.

The first threshold value may be a fixed value such as, for example, 72[dB]. Further, the second threshold value may be a fixed value such as,for example, 54 [dB]. Further, the first threshold value and the secondthreshold value may change dynamically depending on a distance betweenthe user such as the user U1 and the partner performing communicationsuch as the communication partner U2. As an example in which the firstthreshold value and the second threshold value change dynamically, forexample, a “technique of increasing the threshold value by 6 [dB] eachtime the distance is reduced by 0.5 [m] and decreasing the thresholdvalue by 6 [dB] each time the distance is increased by 0.5 [m]” may beused. The distance may be estimated, for example, by performingarbitrary image processing on a captured image captured by the imagingdevice or may be acquired by a distance sensor. In a case in which thedistance is estimated, a process related to the estimation of thedistance may be performed by the information processing apparatusaccording to the present embodiment or may be performed in the externalapparatus of the information processing apparatus according to thepresent embodiment.

Further, the third threshold value and the fourth threshold value may befixed values that are set in advance or may be variable values that canbe changed on the basis of a manipulation of the user or the like.

Further, the type of weight related to the summary decided on the basisof the voice information is not limited to the above example.

For example, it is possible to estimate an emotion (for example, anger,joy, sadness, or the like) on the basis of either or both of the numberof moras and a position of an accent obtained from the voice informationand set the type of weight related to the summary corresponding to theestimated emotion. In a case in which the type of weight related to thesummary corresponding to the estimated emotion is set, the informationprocessing apparatus according to the present embodiment may change astrength of the weight related to the emotion on the basis of, forexample, a change rate of a basic frequency, a change rate of a sound, achange rate of a speaking period, or the like obtained from the voiceinformation.

Here, similarly to the first example described in (a-1), the informationprocessing apparatus according to the present embodiment may decide thetype of weight related to the summary using the tables for specifyingthe type of weight related to the summary as illustrated in FIGS. 7 and8 or decide the weight related to the summary using only the table forspecifying the weight related to the summary illustrated in FIG. 6.

If the weight related to the summary is decided, similarly to the firstexample described in (a-1), the information processing apparatusaccording to the present embodiment sets a weighting to a vocabularycorresponding to the combination indicated by the value “1” among thecombinations of specified types of weight related to the summary andvocabularies with reference to the table for specifying the weightrelated to the summary as illustrated in FIG. 6.

(a-3) Third Example of Setting of Weight Related to Summary: One Exampleof Setting of Weight Related to Summary Based on Execution State ofApplication Indicated by Information Related to Application

The information processing apparatus according to the present embodimentsets the weight related to the summary on the basis of the executionstate of the application.

For example, in a case in which the user U1 manipulates an apparatussuch as a smartphone to activate the scheduling application and checksthe destination, the information processing apparatus according to thepresent embodiment specifies “time” and “place” corresponding toschedule content “changing places (biz)” as the type of weight relatedto the summary from the table for specifying the type of weight relatedto the summary illustrated in FIG. 7 on the basis of the execution stateof the scheduling application. Further, the information processingapparatus according to the present embodiment sets a weighting to avocabulary word corresponding to the combination indicated by the value“1” among the combinations of specified types of weight related to thesummary and vocabulary words with reference to the table for specifyingthe weight related to the summary illustrated in FIG. 6. In a case inwhich the table for specifying the weight related to the summaryillustrated in FIG. 6 is used, the weighting is set to the vocabulary“morning,” “Shibuya,” “when,” “where,” . . . .

Further, the information processing apparatus according to the presentembodiment can decide the type of weight related to the summary on thebasis of, for example, a property of an application being executed asdescribed below and set the weight related to the summary.

-   -   In a case in which a map application is being executed, “time,”        “place,” “personal name,” or the like are decided as the type of        weight related to the summary.    -   In a case in which a transfer guidance application is being        executed: “time,” “place,” “train,” or the like are decided as        types of weight related to the summary.    -   In a case in which an application for smoothly asking questions        about Japan is being executed, “question,” “Japan,” or the like        are decided as the type of weight related to the summary.        (a-4) Fourth Example of Setting of Weight Related to Summary:        One Example of Setting of Weight Related Summary Based on        Manipulation of User Indicated by Manipulation Information of        User Included in Information Related to User

The information processing apparatus according to the present embodimentsets the weight related to the summary on the basis of the manipulationof the user.

For example, the information processing apparatus according to thepresent embodiment selects the type of weight related to the summaryselected by a manipulation of selecting the type of weight related tothe summary (an example of the manipulation of the user) as the type ofweight related to the summary used for the setting of the weight relatedto the summary.

Further, for example, when a predetermined manipulation such as amanipulation of starting the voice recognition related to the summary isperformed, the information processing apparatus according to the presentembodiment may automatically set the type of weight related to thesummary associated with the predetermined manipulation in advance. As anexample, in a case in which the manipulation of starting the voicerecognition related to the summary is performed, “question” or the likeis decided as the type of weight related to the summary.

If the weight related to the summary is decided, similarly to the firstexample described in (a-1), the information processing apparatusaccording to the present embodiment sets a weighting to a vocabularycorresponding to the combination indicated by the value “1” among thecombinations of specified types of weight related to the summary andvocabularies with reference to the table for specifying the weightrelated to the summary as illustrated in FIG. 6.

(a-5) Fifth Example of Setting of Weight Related to Summary

The information processing apparatus according to the present embodimentcan set the weight related to the summary by combining two or more of(a-1) to (a-4) described above.

(b) Example of Summarization Process According to First InformationProcessing Method

For example, a case in which when the user U1 desires to throw trash ata station while moving towards the destination, because there is notrash can at the station, the user U1 asks the communication partner U2about a “reason why there is no trash can at the station” in English(FIGS. 1 and 2) is assumed

Here, in a case in which the communication partner U2 is unable to fullyunderstand English, it is highly likely that the communication partnerU2 is unable to fully understand content which the user U1 is visiting.

In this regard, the information processing apparatus according to thepresent embodiment performs the summarization process according to thefirst information processing method, and summarizes the content of thespeech indicated by the voice information generated by the microphoneconnected to, for example, the eyewear type apparatus illustrated inFIG. 1. The information processing apparatus according to the presentembodiment summarizes, for example, a character string indicated byvoice text information based on the voice information as describedabove.

More specifically, the information processing apparatus according to thepresent embodiment summarizes the content of the speech according to anobjective function using the weight related to the summary set accordingto, for example, the process described in (a) as indicated in thefollowing Formula 1.

[Math.  1]                                        $\begin{matrix}{D = {\arg\limits_{y}\mspace{14mu}{\max\left( {\sum\limits_{a}{\sum\limits_{W}{\sum\limits_{y}{a_{i} \cdot W_{{jy}_{i}} \cdot z_{y_{i}}}}}} \right)}}} & \left( {{Formula}\mspace{14mu} 1} \right)\end{matrix}$

Here, “W” in Formula 1 indicates the weight related to the summary.Further, “a_(i)” in Formula 1 indicates a parameter for adjusting acontribution rate of each of the weights related to summary, and has areal number of, for example, 0 to 1. Further, “z_(yi)” in Formula 1indicates a binary variable indicating “1” if a phrase y_(i) is includedand “0” if the phrase y_(i) is not included.

Further, the information processing apparatus according to the presentembodiment is not limited to the method using the objective functionusing the weight related to the summary indicated by Formula 1, and anarbitrary method capable of summarizing the content of the speech usingthe set weight related to the summary can be used.

FIGS. 3A and 3B illustrate an example of a result of the summarizationprocess according to the first information processing method. FIG. 3Aillustrates an example of the content of the speech before beingsummarized. Further, FIG. 3B illustrates an example of the summarizedcontent of the speech, and FIG. 3C illustrates another example of thesummarized content of the speech.

As the content of the speech is summarized as illustrated in FIG. 3B,the content of the speech is more simplified than before the content ofthe speech is summarized. Therefore, even in a case in which thecommunication partner U2 is unable to fully understand English, sincethe content of the speech is summarized as illustrated in FIG. 3B, it ishighly likely that the communication partner U2 can understands thecontent which the user U1 is visiting.

Further, FIG. 3C illustrates an “example in which the informationprocessing apparatus according to the present embodiment performsfurther morphological analysis on the summarization result illustratedin FIG. 3B and sets divisional texts obtained by dividing thesummarization result illustrated in FIG. 3B in units in which morphemesbased on a result of performing morphological analysis are combined asthe summarized content of the speech.”

For example, in a case in which the language of the character stringindicated by the voice text information corresponding to the content ofthe speech is Japanese, the information processing apparatus accordingto the present embodiment generates the divisional texts in units inwhich main parts of speech (a noun, a verb, an adjective, and an adverb)and the other morphemes are combined. Further, for example, in a case inwhich the language of the character string indicated by the voice textinformation corresponding to the content of the speech is English, theinformation processing apparatus according to the present embodimentfurther sets 5W1H as the divisional text.

As the content of the speech is summarized as illustrated in FIG. 3C,the content of the speech is simplified more than the summarizationresult illustrated in FIG. 3B. Therefore, even in a case in which thecommunication partner U2 is unable to sufficiently understand English,since the content of the speech is summarized as illustrated in FIG. 3C,it is possible to further increase the possibility that thecommunication partner U2 can understands the content which the user U1is visiting.

(c) An Example of Translation Process

The information processing apparatus according to the present embodimentmay further translate, for example, the content of the speech summarizedaccording to the summarization process described in (b) into anotherlanguage. As described above, the information processing apparatusaccording to the present embodiment translates a first languagecorresponding to the speech into a second language different from thefirst language.

For example, the information processing apparatus according to thepresent embodiment identifies a position at which the user U1 islocated, and translates the summarized content of the speech into anofficial language in a case in which the language of the characterstring indicated by the voice text information corresponding to thecontent of the speech is different from the official language at thespecified position. The position at which the user U1 is located is, forexample, specified on the basis of position information acquired from awearable apparatus worn by the user U1 such as the eyewear typeapparatus illustrated in FIG. 1, a communication apparatus such as asmartphone possessed by the user U1, or the like. The positioninformation is, for example, data indicating a detection result of adevice capable of specifying a position such as a global navigationsatellite system (GNSS) device (or data indicating an estimation resultof a device capable of estimating a position according to an arbitrarymethod).

Further, for example, in a case in which the language of characterstring indicated by the voice text information corresponding to thecontent of the speech is different from the set language, theinformation processing apparatus according to the present embodiment maytranslate the summarized content of the speech into the set language.

The information processing apparatus according to the present embodimenttranslates the summarized content of the speech into another languageaccording to a process of an arbitrary algorithm capable of translatinginto another language.

FIGS. 4A and 4B illustrate an example of a result of the translationprocess according to the present embodiment. FIG. 4A illustrates thesummarization result illustrated in FIG. 3C as an example of thesummarized content of the speech before being translated. Further, FIG.4B illustrates an example of a translation result of translating thesummarization result illustrated in FIG. 3C into Japanese as an exampleof content in which the summarization result illustrated in FIG. 3C istranslated into another language according to the translation process.Hereinafter, the translation result obtained by translating thedivisional text such as the summarization result illustrated in FIG. 3Cis also referred to as “divisional translation text.”

As illustrated in FIG. 4B, the summarized content of the speech istranslated into Japanese which is the native language of thecommunication partner U2, and thus the possibility that thecommunication partner U2 can understand the content which the user U1 isvisiting can be further increased as compared with a case in which thesummarized content of the speech is not translated.

(d) Example of Notification Control Process According to SecondInformation Processing Method

The information processing apparatus according to the present embodimentcauses a notification of the content of the speech which is indicated bythe voice information and summarized according to the summarizationprocess described in (b) to be given. Further, the translation processdescribed in (c) is further performed, and in a case in which thesummarized content of the speech is translated into another language,the information processing apparatus according to the present embodimentcauses a notification of the translation result to be given.

As described above, the information processing apparatus according tothe present embodiment may cause a notification of the summarizedcontent of the speech (or the translation result) to be given as thenotification content according to, for example, either or both of thenotification based on the visual method and the notification based onthe auditory method.

FIG. 5 illustrates an example of a result of the notification controlprocess according to the present embodiment. FIG. 5 illustrates an“example in which an auditory notification of the translation result isgiven by outputting a voice indicating the translation result from thevoice output device connected to the eyewear type apparatus worn by theuser U1.” Further, FIG. 5 illustrates an example in which a notificationof the translation result illustrated in of FIG. 4B is given.

FIG. 5 illustrates an example in which sound pressure of a partcorresponding to a speech part in which the sound pressure is strong (apart of “why” illustrated in FIG. 5) is caused to be stronger than theother parts on the basis of the voice information.

Further, FIG. 5 illustrates an example in which, when the voiceindicating the translation result is output, a notification of thedelimiter of the divisional text is given by inserting a sound feedbackindicated by a reference numeral “S” in FIG. 5.

Further, the example of the notification implemented according to thenotification control process according to the second informationprocessing method is not limited to the example illustrated in FIG. 5.Another example of the notification implemented according to thenotification control process according to the second informationprocessing method will be described later.

For example, as illustrated in FIG. 5, the summarized content of thespeech translated into Japanese which is the native language of thecommunication partner U2 (the translation result) is output as thenotification content from the voice output device by voice, and thus itis easier to cause the communication partner U2 to understand thecontent which the user U1 is visiting.

The use case to which the information processing method of the presentembodiment is applied includes the use case of the “conversationsupport” (including the case in which translation is performed).Further, needless to say, the use case to which the informationprocessing method of the present embodiment is applied is not limited tothe “conversation support.”

[3] Process Related to Information Processing Method According toPresent Embodiment

Next, a process according to the information processing method of thepresent embodiment will be described in further detail. Thesummarization process according to the first information processingmethod, the translation process according to the present embodiment, andthe notification control process according to the second informationprocessing method will be described below.

[3-1] Summarization Process According to First Information ProcessingMethod

The information processing apparatus according to the present embodimentsummarizes the content of the speech indicated by the voice informationbased on the speech of the user on the basis of the informationindicating the weight related to the summary.

As described above, the weight related to the summary is set on thebasis of one or more of, for example, the voice information, the stateof the user, the execution state of the application, and themanipulation of the user. Further, as described above, the informationprocessing apparatus according to the present embodiment summarizes thecontent of the speech according to, for example, an objective functionusing the weight related to the summary set as indicated in Formula 1.

Further, for example, the information processing apparatus according tothe present embodiment can perform one or more of the followingprocesses (1) to (3) as the summarization process.

(1) First Example of Summarization Process: Start Timing ofSummarization Process

The information processing apparatus according to the present embodimentperforms the summarization process in a case in which a setpredetermined condition is satisfied.

Examples of the start condition of the summarization process accordingto the present embodiment include the following examples.

-   -   A conditions related to a non-speaking period in which a        non-speech state is continued. A condition related to a state of        the voice recognition for acquiring the content of the speech        from the voice information.    -   A conditions related to the content of the speech    -   A conditions related to an elapsed time since the voice        information is obtained

FIGS. 9A, 9B, 9C, 9D, 9E, and 9F are explanatory diagrams for describingan example of the summarization process according to the firstinformation processing method, and illustrates an overview of a starttiming of the summarization process. An example of a process in eachstart condition will be described below with reference to FIGS. 9A 9B,9C, 9D, 9E, and 9F.

(1-1) First Example of Start Condition: Example in a Case in which StartCondition is Condition Related to Non-Speaking Period

The condition related to the non-speaking period is, for example, acondition related to a length of the non-speaking period. In a case inwhich the predetermined start condition is a condition related to thenon-speaking period, the information processing apparatus according tothe present embodiment determines that the start condition is satisfiedin a case in which the non-speaking period exceeds a set predeterminedperiod or in a case in which the non-speaking period it is equal to orlonger than a set predetermined period.

Here, the period according to the first example of the start conditionmay be a preset fixed period or may be a variable period that can bechanged on the basis of the manipulation of the user or the like.

Referring to FIG. 9A, a “silent interval” illustrated of FIG. 9Acorresponds to the non-speaking period.

The information processing apparatus according to the presentembodiment, for example, detects a voice interval in which there is avoice on the basis of the voice information. Then, in a case in whichthe silent interval is detected during a set time or more after thevoice section is detected or in a case in which the silent interval of aset time or more is detected, the information processing apparatusaccording to the present embodiment regards it as a start trigger of thesummarization process (hereinafter referred to as a “summarizationtrigger”) and starts the summarization process.

(1-2) Second Example of Start Condition: Example in a Case in whichStart Condition is First Condition Related to a State of VoiceRecognition

A first condition related to the state of the voice recognition may be acondition related to detection of a voice recognition stop request. In acase in which a predetermined start condition is the first conditionrelated to the state of the voice recognition, the informationprocessing apparatus according to the present embodiment determines thatthe start condition is satisfied on the basis of the detection of thevoice recognition stop request. The information processing apparatusaccording to the present embodiment determines that the start conditionis satisfied, for example, in a case in which the voice recognition stoprequest is detected.

Referring to FIG. 9B, for example, in a case in which the voicerecognition stop request including a voice recognition stop commandbased on a “manipulation of stopping the voice recognition” illustratedin FIG. 9B after the voice recognition is started on the basis of a“voice recognition start manipulation” illustrated in FIG. 9B, theinformation processing apparatus according to the present embodimentregards it as the summarization trigger and starts the summarizationprocess. Here, the voice recognition start manipulation and the voicerecognition stop manipulation may be, for example, a manipulation on anarbitrary user interface (UI) related to the voice recognition.

Further, the voice recognition stop request according to the presentembodiment is not limited to one obtained on the basis of the voicerecognition stop manipulation. For example, the voice recognition stoprequest may be generated by an apparatus or the like performing thevoice recognition process, for example, in a case in which an erroroccurs during the voice recognition process, in a case in which aninterrupt process is performed during the voice recognition process, orthe like.

(1-3) Third Example of Start Condition: Example in a Case in which StartCondition is Second Condition Related to State of Voice Recognition

A second condition related to the state of the voice recognition is acondition related to completion of the voice recognition. In a case inwhich a predetermined start condition is the second condition related tothe state of the voice recognition, the information processing apparatusaccording to the present embodiment determines that the start conditionis satisfied on the basis of detection of the completion of the voicerecognition. The information processing apparatus according to thepresent embodiment determines that the start condition is satisfied, forexample, in a case in which the completion of the voice recognition isdetected.

Referring to FIG. 9C, for example, in a case in which a result of thevoice recognition process is obtained as indicated by “voice recognitionresult acquisition” in FIG. 9C, the information processing apparatusaccording to the present embodiment regards it as the summarizationtrigger and starts the summarization process.

(1-4) Fourth Example of Start Condition: Example in a Case in whichStart Condition is First Condition Related to Content of Speech

A first condition related to the content of the speech may be acondition related to detection of a predetermined word from the contentof the speech indicated by the voice information. In a case in which apredetermined start condition is the first condition related to thecontent of the speech, the information processing apparatus according tothe present embodiment determines that the starts condition is satisfiedon the basis of detection of a predetermined word from the content ofthe speech indicated by the voice information. The informationprocessing apparatus according to the present embodiment determines thatthe start condition is satisfied, for example, in a case in which apredetermined word is detected from the content of the speech indicatedby the voice information.

The predetermined word related to the first condition related to thecontent of the speech may be, for example, a word called a filler word.The predetermined word related to the first condition related to thecontent of the speech may be a preset fixed word which is unable to beadded, deleted, changed, or the like or may be added, deleted, orchanged on the basis of a manipulation of the user.

Referring to FIG. 9D, “well” illustrated in FIG. 9D corresponds to anexample of the filler word (an example of a predetermined word).

The information processing apparatus according to the present embodimentstarts the summarization process using, for example, a case in which thefiller word is detected from the character string indicated by the voicetext information obtained on the basis of the voice information as thesummarization trigger.

(1-5) Fifth Example of Start Condition: Example in a Case in which StartCondition is Second Condition Related to Content of Speech

A second condition related to the content of the speech is a conditionrelated to detection of hesitation to speak from the content of thespeech indicated by the voice information. In a case in which thepredetermined start condition is a second condition related to thecontent of the speech, the information processing apparatus according tothe present embodiment determines that the start condition is satisfiedon the basis of the detection of the hesitation to speak on the basis ofthe voice information. The information processing apparatus according tothe present embodiment determines that the start condition is satisfied,for example, in a case in which the hesitation to speak is detected onthe basis of the voice information.

The information processing apparatus according to the present embodimentdetects the hesitation to speak on the basis of the voice informationaccording to an arbitrary method capable of detecting the hesitation tospeak on the basis of the voice information or estimating the hesitationto speak on the basis of the voice information such as, for example, a“method of detecting a voiced pause (including delay of a syllable aswell) from the voice information” or a “method of detecting a wordassociated with the hesitation to speak from a character stringindicated by the voice text information obtained on the basis of thevoice information.”

Referring to FIG. 9E, the information processing apparatus according tothe present embodiment starts the summarization process using, forexample, a case in which it is estimated that there is hesitation tospeak as the summarization trigger.

(1-6) Sixth Example of Start Condition: Example in a Case in which StartCondition is Condition Related to Elapsed Time after Voice Informationis Obtained

A condition related to the elapsed time since the voice information isobtained may be a condition on the length of the elapsed time. In a casein which the predetermined start condition is a condition related to theelapsed time since the voice information is obtained, the informationprocessing apparatus according to the present embodiment determines thatthe start condition is satisfied in a case in which the elapsed timeexceeds a set predetermined period or in a case in which the elapsedtime is a set predetermined period or more.

Here, the period according to the sixth example of the start conditionmay be a preset fixed period or may be a variable period that can bechanged on the basis of the manipulation of the user or the like.

Referring to of FIG. 9F, the information processing apparatus accordingto the present embodiment starts the summarization process using, forexample, a case in which a set certain time elapses since theacquisition of the voice information is detected as the summarizationtrigger.

(1-7) Seventh Example of Start Condition

The start condition may be a condition in which two or more of the startconditions according to the first to sixth examples described in (1-1)to (1-6) are combined. The information processing apparatus according tothe present embodiment starts the summarization process using, forexample, a case in which any one of the combined start conditions issatisfied as the summarization trigger.

(2) Second Example of Summarization Process: Exception Process in whichSummarization Process is not Performed

The information processing apparatus according to the present embodimentdoes not perform the summarization process in a case in which a setsummarization process exclusion condition (hereinafter referred to as“summarization exclusion condition”) is determined to be satisfied.

The summarization exclusion condition according to the presentembodiment may be, for example, a condition related to detection of agesture. The information processing apparatus according to the presentembodiment determines that the summarization exclusion condition issatisfied in a case in which a set predetermined gesture is detected.

The predetermined gesture related to the summarization exclusioncondition may be a preset fixed gesture or may be added, deleted orchanged on the basis of the manipulation of the user or the like. Theinformation processing apparatus according to the present embodimentdetermines whether or not the predetermined gesture related to thesummarization exclusion condition is performed, for example, byperforming image processing on a captured image obtained according toimaging by an imaging device, estimating a motion on the basis of adetection result of a motion sensor such as an acceleration sensor or anangular velocity sensor, or the like.

Further, the summarization exclusion condition according to the presentembodiment is not limited to the conditions related to the detection ofthe gesture described above.

For example, the summarization exclusion condition according to thepresent embodiment may be an arbitrary condition set as thesummarization exclusion condition such as “detection of a manipulationof disabling a function of performing the summarization process” such as“pressing of a button for disabling the function of performing thesummarization process,” a “state in which the processing load of theinformation processing apparatus according to the present embodiment islarger than a set threshold value.”

(3) Third Example of Summarization Process: Process of DynamicallyChanging Summarization Level

The information processing apparatus according to the present embodimentchanges a summarization level of the content of the speech (or asummarization degree of the content of the speech; The same applieshereinafter) on the basis of either or both of the speaking periodspecified on the basis of the voice information and the number ofcharacters specified on the basis of the voice information. In otherwords, the information processing apparatus according to the presentembodiment changes the summarization level of the content of the speechon the basis of at least one of the speaking period specified on thebasis of the voice information and the number of characters specified onthe basis of the voice information.

The information processing apparatus according to the present embodimentchanges the summarization level of the content of the speech, forexample, by limiting the number of characters indicated by thesummarized content of the speech. The information processing apparatusaccording to the present embodiment limits the number of charactersindicated by the summarized content of the speech by causing the numberof characters indicated by the summarized content of the speech not toexceed a set upper limit value. Since the number of characters indicatedby the summarized content of the speech is limited, it is possible toautomatically reduce the number of characters indicated by thesummarized content of the speech, that is, a summarization amount.

Here, the speaking period is specified, for example, by detecting thevoice interval in which there is a voice on the basis of the voiceinformation. Further, the number of characters corresponding to thespeech is specified by counting the number of characters of thecharacter string indicated by the voice text information based on thevoice information.

In a case in which the summarization level of the content of the speechis changed on the basis of the speaking period, the informationprocessing apparatus according to the present embodiment changes thesummarization level of the content of the speech, for example, in a casein which the speaking period exceeds a set predetermined period or in acase in which the speaking period is a set predetermined period or more.Here, the period in the case in which the summarization level of thecontent of the speech is changed on the basis of the speaking period maybe a preset fixed period or may be a variable period that can be changedon the basis of the manipulation of the user or the like.

Further, in a case in which the summarization level of the content ofthe speech is changed on the basis of the number of characters specifiedon the basis of the voice information, the information processingapparatus according to the present embodiment changes the summarizationlevel of the content of the speech, for example, in a case in which thenumber of characters is larger than a set threshold value or in a casein which the number of characters is a set threshold value or more.Here, the threshold value in the case in which the summarization levelof the content of the speech is changed on the basis of the number ofcharacters specified on the basis of the voice information may be apreset fixed threshold value or may be a variable threshold value thatcan be changed on the basis of the manipulation of the user or the like.

[3-2] Translation Process According to Present Embodiment

As described in (c), the information processing apparatus according tothe present embodiment can further perform the translation process oftranslating the content of the speech summarized according to thesummarization process according to the first information processingmethod into another language. As described above, the informationprocessing apparatus according to the present embodiment translates thefirst language corresponding to the speech into the second languagedifferent from the first language.

Further, in the translation process, a degree of reliability of thetranslation result may be set for each translation unit.

The translation unit is a unit in which translation is performed in thetranslation process. The translation unit may be, for example, a fixedunit set for each word, every one or two or more clauses, or the like.Further, for example, the translation unit may be dynamically set inaccordance with the language (first language) corresponding to thespeech. Further, the translation unit may be changeable on the basis of,for example, a setting manipulation of the user.

The degree of reliability of the translation result is, for example, anindex indicating certainty of the translation result and indicated by,for example, a value of 0 [%] (indicating that the degree of reliabilityis lowest) to 100 [%] (the degree of reliability is highest). The degreeof reliability of the translation result can be obtained using a resultof arbitrary machine learning such as a result of machine learning usinga result of feedback with respect to the translation result. Further,the degree of reliability of the translation result is not limited toone obtained using the machine learning and may be obtained according toan arbitrary method capable of obtaining the certainty of thetranslation result.

Further, the information processing apparatus according to the presentembodiment can perform either or both of the following processes (i) and(ii) as the translation process.

(i) First Example of Translation Process: Exception Process in whichTranslation Process is not Performed

In a case in which a set translation process exclusion condition isdetermined to be satisfied, the information processing apparatusaccording to the present embodiment does not perform the translationprocess.

The exclusion condition of the translation process according to thepresent embodiment may be, for example, a condition related to detectionof a gesture. The information processing apparatus according to thepresent embodiment determines that the translation process is satisfiedin a case in which a set predetermined gesture is detected.

The predetermined gesture related to the translation process may be afixed gesture set in advance or may be added, deleted or changed on thebasis of the manipulation of the user or the like. The fixed gesture setin advance may be, for example, a body gesture, a hand gesture, or thelike related to non-verbal communication such as a hand sign. Theinformation processing apparatus according to the present embodimentdetermines whether or not the predetermined gesture related to thetranslation process is performed, for example, by performing imageprocessing on a captured image obtained according to imaging by animaging device or estimating a motion on the basis of a detection resultof a motion sensor such as an acceleration sensor or an angular velocitysensor.

Further, the translation process exclusion condition according to thepresent embodiment is not limited to the condition related to thedetection of the gesture described above.

For example, the translation process exclusion condition according tothe present embodiment may be an arbitrary condition set as thetranslation process exclusion condition such as “detection of amanipulation of disabling a function of performing the translationprocess” such as “pressing of a button for disabling the function ofperforming the translation process,” a “state in which the processingload of the information processing apparatus according to the presentembodiment is larger than a set threshold value.” Further, thetranslation process exclusion condition according to the presentembodiment may be the same condition as or a different condition fromthe summarization exclusion condition according to the presentembodiment described above.

(ii) Second Example of Translation Process: Processing in Retranslation

The information processing apparatus according to the present embodimentcan also retranslate the content translated into another language intothe language before the translation.

In the information processing apparatus according to the presentembodiment retranslates the content translated into another languageinto the language before the translation, for example, in a case inwhich a manipulation of performing a retranslation process such aspressing of a button for performing a retranslation is detected.

Further, the retranslation trigger is not limited to the detection ofthe manipulation of performing the retranslation process. For example,the information processing apparatus according to the present embodimentcan automatically perform the retranslation on the basis of the degreeof reliability of the translation result set for each translation unit.The information processing apparatus according to the present embodimentperforms the retranslation using, for example, a case in which there isa degree of reliability equal to or less than a set threshold value orthere is a degree of reliability smaller than the threshold value amongthe degrees of reliability of the translation result set for eachtranslation unit as the retranslation trigger.

Further, in a case in which the content translated into another languageis retranslated into the language before the translation, theinformation processing apparatus according to the present embodiment mayperform the summarization process using a retranslation result.

As an example, the information processing apparatus according to thepresent embodiment includes a word included in content after theretranslation in the summarized content of the speech, for example, in acase in which there is a word included in the content after theretranslation in the content of the speech indicated by the voiceinformation acquired after the retranslation. Since the summarizationprocess using the retranslation result is performed, for example, an“adjustment of causing the same word as before the retranslation not tobe deleted in a summary corresponding to a current speech in a case inwhich the same word as before the retranslation appears in contentspoken by the user” is implemented.

[3-3] Notification Control Process According to Second InformationProcessing Method

The information processing apparatus according to the present embodimentcauses a notification of the content of the speech indicated by thevoice information summarized according to the summarization processaccording to the first information processing method to be given.

In a case in which the summarized content of the speech is translatedinto another language according to the translation process according tothe present embodiment as described above, the information processingapparatus according to the present embodiment causes a notification ofthe translation result to be given.

Further, as described above, the information processing apparatusaccording to the present embodiment causes a notification of thenotification content to be given, for example, according to either orboth the notification based on the visual method and the notificationbased on the auditory method.

FIG. 10 is an explanatory diagram illustrating an example ofnotification by a visual method implemented by a notification controlprocess according to the second information processing method. FIG. 10illustrates an example in which the information processing apparatusaccording to the present embodiment causes the translation result to bedisplayed on the display screen of the smartphone.

Further, the information processing apparatus according to the presentembodiment can perform one or more of the following processes (I) to(VII) as the notification control process. A case in which theinformation processing apparatus according to the present embodimentcauses a notification of the translation result to be given will bedescribed as an example. Further, the information processing apparatusaccording to the present embodiment can cause a notification of thesummarized content of the speech before the translation to be given,similarly to the case of causing a notification of the translationresult to be given.

FIGS. 11, 12, 13, 14, 15, 16A, 16B, 16C, 16D, 16E, 16F, 17, 18, 19A,19B, 20A, 20B, 21A, 21B and 21C are explanatory diagrams for describingan example of the notification control process according to the secondinformation processing method. An example of the notification controlprocess according to the second information processing method will bedescribed below with reference to FIGS. 11, 12, 13, 14, 15, 16A, 16B,16C, 16D, 16E, 16F, 17, 18, 19A, 19B, 20A, 20B, 21A, 21B and 21C asappropriate.

(I) First Example of Notification Control Process: Word Order Notice ofTranslation Language

The information processing apparatus according to the present embodimentcauses a notification of the translation result to be given in a wordorder corresponding to another language which is translated.

For example, in the summarization process, when the content of thespeech is summarized into the divisional text illustrated in FIG. 3C, ina case in which another language is English, the information processingapparatus according to the present embodiment causes a notification ofthe translation result to be given in the following order:

-   -   noun    -   verb    -   adjective    -   adverb    -   others

Further, for example, in the summarization process, when the content ofthe speech is summarized into the divisional text illustrated in FIG.3C, in a case in which another language is Japanese, the informationprocessing apparatus according to the present embodiment causes anotification of the translation result to be given in the followingorder:

-   -   verb    -   noun    -   adjective    -   adverb    -   others

Since a notification of the translation result is caused to be given inthe word order corresponding to another language which is translated asdescribed above, for example, it is possible to change the word order ofthe translation result illustrated in FIG. 4B and the word order theauditory notification as illustrated in FIG. 5.

Here, the word order corresponding to another language which istranslated may be a preset fixed word order or may be changeable on thebasis of the manipulation of the user or the like.

(II) Second Example of Notification Control Process: NotificationControl Process Based on a Degree of Reliability of Each TranslationUnit

As described above, in the translation process, the degree ofreliability of the translation result can be set for each translationunit. In the translation process, in a case in which the degree ofreliability of the translation result is set for each translation unit,the information processing apparatus according to the present embodimentcauses a notification of the translation result to be given on the basisof the degree of reliability of each translation unit in the summarizedcontent of the speech.

The information processing apparatus according to the present embodimentcauses a notification of the translation result to be given on the basisof the degree of reliability of each translation unit by performing, forexample, either or both of the following processes (II-1) and (II-2).

(II-1) First Example of Notification Control Process Based on Basis ofDegree of Reliability of Each Translation Unit

The information processing apparatus according to the present embodimentcauses a notification of a translation result with a high degree ofreliability to be given with priority.

For example, in a case in which a visual notification is given bycausing the translation result to be displayed on the display screen ofthe display device, the information processing apparatus according tothe present embodiment implements a preferential notification of thetranslation result with the high degree of reliability depending on adisplay method. Further, in a case in which an auditory notification ofthe translation result is given through a voice from the voice outputdevice, the information processing apparatus according to the presentembodiment may implement the preferential notification of thetranslation result with the high degree of reliability in accordancewith, for example, a notification order.

An example of the notification implemented by the notification controlprocess based on the degree of reliability of each translation unitaccording to a first example will be described below using a case inwhich a visual notification is given by causing the translation resultto be displayed on the display screen of the display device as anexample.

FIG. 11 illustrates a first example of causing the translation result tobe displayed on the display screen of the display device, andillustrates an example of a case in which a notification of thetranslation result with a high degree of reliability is given withpriority. In the example illustrated in FIG. 11, each of“recommendation,” “sightseeing,” “route,” “tell,” and “Asakusa”corresponds to the translation results of each translation unit.Further, FIG. 11 illustrates an example in which the degree ofreliability to be set decreases in the order of “recommendation,”“sightseeing,” “route,” “tell,” and “Asakusa.”

The information processing apparatus according to the present embodimentcauses the translation result of each translation unit to be displayedso that the translation result of each translation unit ishierarchically displayed in the descending order of the degrees ofreliability, for example, as illustrated in A of FIG. 11.

Here, the hierarchical display is implemented by, for example, athreshold value process using the degree of reliability of eachtranslation unit and one or more threshold values related to decision ofa layer on which display is performed. Here, the threshold value relatedto the hierarchical display may be a preset fixed value or may be avariable value that can be changed on the basis of the manipulation ofthe user or the like.

In a case in which the translation results of every two or moretranslation units is displayed on the same layer as a result of thethreshold value process, the information processing apparatus accordingto the present embodiment causes the translation results of every two ormore translation units to be displayed in a set predetermined order, forexample, “in which the translation results are arranged in thedescending order of the degrees of reliability from the left to theright in a region of the display screen corresponding to the layer.”

Further, in a case in which there are a plurality of translation resultsin which the degree of reliability is larger than a predeterminedthreshold value or a plurality of translation results in which thedegree of reliability is equal to or larger than a predeterminedthreshold value as a result of the threshold value process, theinformation processing apparatus according to the present embodiment maycause a plurality of existing translation results to be displayedtogether in a predetermined region of the display screen, for example,as illustrated in B of FIG. 11. Here, as the predetermined thresholdvalue, one or more threshold values among one or more threshold valuesused for the threshold value process may be used. Further, thepredetermined region may be, for example, a “region of the displayscreen corresponding to a layer associated with a threshold valueprocess using a predetermined threshold value.”

For example, as the display is performed as illustrated in FIG. 11, itis implemented that “the translation result of each translation unit inwhich a high degree of reliability (corresponding to a score) is set inthe translation process is displayed on a high level, and in a case inwhich the degree of reliability exceeds a predetermined threshold value,the translation results of each translation unit are displayedtogether.” Further, needless to say, the example of the display in acase in which a notification of the translation result with the highdegree of reliability is given with priority is not limited to theexample illustrated in FIG. 11.

(II-2) Second Example of Notification Control Process Based on Degree ofReliability of Each Translation Unit

The information processing apparatus according to the present embodimentcauses a notification of the translation result to be given so thatemphasis is performed in accordance with the degree of reliability.

For example, in a case in which the visual notification is given bycausing the translation result to be displayed on the display screen ofthe display device, the information processing apparatus according tothe present embodiment implements a notification emphasized inaccordance with the degree of reliability depending on the displaymethod. Further, in a case in which the auditory notification is giventhrough a voice from the voice output device, the information processingapparatus according to the present embodiment may implement anotification emphasized in accordance with the degree of reliability,for example, by changing the sound pressure, the volume, or the like ofthe voice on the basis of the degree of reliability.

An example of the notification implemented by the notification controlprocess based on the degree of reliability of each translation unitaccording to a second example will be described below using a case inwhich a visual notification is given by causing the translation resultto be displayed on the display screen of the display device as anexample.

The information processing apparatus according to the present embodimentcauses the translation result to be emphatically displayed in accordancewith the degree of reliability by, for example, “causing eachtranslation result of each translation unit to be displayed with a sizecorresponding to the degree of reliability.”

FIG. 12 illustrates a second example in which the translation result isdisplayed on the display screen of the display device, and illustrates afirst example in which the translation result is emphatically displayedin accordance with the degree of reliability. In the example illustratedin FIG. 12, each of “recommendation,” “sightseeing,” “route,” “tell,”and “Asakusa” corresponds to the translation results of each translationunit. Further, FIG. 12 illustrates an example in which the degree ofreliability to be set decreases in the order of “recommendation,”“sightseeing,” “route,” “tell,” and “Asakusa.”

Further, FIG. 12 illustrates an example in which the informationprocessing apparatus according to the present embodiment further causeseach translation result of each translation unit to be displayed withthe size corresponding to the degree of reliability in addition to thenotification control process based on the degree of reliability of eachtranslation unit according to the first example. Further, needless tosay, “in a case in which the notification control process based on thedegree of reliability of each translation unit according to the secondexample is performed, the information processing apparatus according tothe present embodiment may cause a notification of the translationresult with the high degree of reliability not to be given with priorityunlike the hierarchical display as illustrated in FIG. 11.”

In the information processing apparatus according to the presentembodiment causes each translation result of each translation unit to bedisplayed with the size corresponding to the degree of reliability, forexample, as illustrated in A of FIG. 12. The information processingapparatus according to the present embodiment causes each translationresult of each translation unit to be displayed with the sizecorresponding to the degree of reliability with reference to, forexample, a “table (or a database) in which the degree of reliability isassociated with the display size for causing the translation result ofeach translation unit to be displayed on the display screen.”

For example, as the display is performed as illustrated in FIG. 12, itis implemented that “the translation result of each translation unit inwhich a high degree of reliability (corresponding to a score) is set inthe translation process is displayed on a high level, and the size ischanged so that the translation result of each translation unitdisplayed on the higher level is noticeable.” Further, needless to say,the example of the display in a case in which each translation result ofeach translation unit is caused to be displayed with the sizecorresponding to the degree of reliability is not limited to the exampleillustrated in FIG. 12.

Further, for example, the information processing apparatus according tothe present embodiment may cause the translation result to beemphatically displayed in accordance with the degree of reliability by,for example, “causing each translation result of each translation unitto be displayed such that the translation result with the high degree ofreliability is displayed on a front side in the display screen.”

FIG. 13 illustrates a third example in which the translation result isdisplayed on the display screen of the display device, and illustrates asecond example in which the translation result is emphatically displayedin accordance with the degree of reliability. In the example illustratedin FIG. 13, each of “recommendation,” “sightseeing,” “route,” “tell,”and “Asakusa,” . . . corresponds to the translation results of eachtranslation unit. Further, FIG. 13 illustrates an example in which thedegree of reliability to be set decreases in the order of“recommendation,” “sightseeing,” “route,” “tell,” and “Asakusa,” . . . .

Further, FIG. 13 illustrates an example in which the informationprocessing apparatus according to the present embodiment further causesthe translation result with the high degree of reliability to bedisplayed on a front side in the display screen in addition to thenotification control process based on the degree of reliability of eachtranslation unit according to the first example. Further, as describedabove, needless to say, “in a case in which the notification controlprocess based on the degree of reliability of each translation unitaccording to the second example is performed, the information processingapparatus according to the present embodiment may cause a notificationof the translation result with the high degree of reliability not to begiven with priority unlike the hierarchical display as illustrated inFIG. 11.”

In the information processing apparatus according to the presentembodiment causes the translation result with the high degree ofreliability to be displayed on a front side in the display screen, forexample, as illustrated in A of FIG. 13. The information processingapparatus according to the present embodiment causes each translationresult of each translation unit to be displayed such that thetranslation result with the high degree of reliability is displayed on afront side in the display screen with reference to, for example, a“table (or a database) in which the degree of reliability is associatedwith a coordinate value in a depth direction when the translation resultof each translation unit is caused to be displayed on the displayscreen.”

For example, as the display is performed as illustrated in FIG. 13, itis implemented “that the translation result of each translation unit inwhich a high degree of reliability (corresponding to a score) is set inthe translation process is displayed on a front side in the depthdirection in the display screen, and thus the translation result of eachtranslation unit in which the high degree of reliability is set iscaused to be noticeable.” Further, needless to say, the example of thedisplay in a case in which each translation result of each translationunit is caused to be displayed such that the translation result with thehigh degree of reliability is displayed on a front side in the displayscreen is not limited to the example illustrated in FIG. 13.

Further, the information processing apparatus according to the presentembodiment may cause the translation result to be emphatically displayedin accordance with the degree of reliability by, for example, “causingeach translation result of each translation unit to be displayed ineither or both of a color corresponding to the degree of reliability andtransparency corresponding to the degree of reliability.”

FIG. 14 illustrates a fourth example in which the translation result isdisplayed on the display screen of the display device, and illustrates athird example in which the translation result is emphatically displayedin accordance with the degree of reliability. In the example illustratedin FIG. 14, each of “recommendation,” “sightseeing,” “route,” “tell,”and “Asakusa” corresponds to the translation results of each translationunit. Further, FIG. 14 illustrates an example in which the degree ofreliability to be set decreases in the order of “recommendation,”“sightseeing,” “route,” “tell,” and “Asakusa.”

Further, FIG. 14 illustrates an example in which the informationprocessing apparatus according to the present embodiment further causeseach translation result of each translation unit to be displayed ineither or both of a color corresponding to the degree of reliability andtransparency corresponding to the degree of reliability in addition tothe notification control process based on the degree of reliability ofeach translation unit according to the first example. Further, asdescribed above, needless to say, “in a case in which the notificationcontrol process based on the degree of reliability of each translationunit according to the second example is performed, the informationprocessing apparatus according to the present embodiment may cause anotification of the translation result with the high degree ofreliability not to be given with priority unlike the hierarchicaldisplay as illustrated in FIG. 11.”

The information processing apparatus according to the present embodimentcauses each translation result of each translation unit to be displayedin a color corresponding to the degree of reliability, for example, asillustrated in A of FIG. 14. Further, the information processingapparatus according to the present embodiment may cause each translationresult of each translation unit to be displayed with transparencycorresponding to the degree of reliability. Further, the informationprocessing apparatus according to the present embodiment can also causeeach translation result of each translation unit to be displayed in bothof the color corresponding to the degree of reliability and thetransparency corresponding to the degree of reliability.

The information processing apparatus according to the present embodimentcauses each translation result of each translation unit to be displayedin either or both of the color corresponding to the degree ofreliability and the transparency corresponding to the degree ofreliability with reference to, for example, a “table (or a database) inwhich the degree of reliability, the color for causing the translationresult of each translation unit to be displayed, and the transparencyfor causing the translation result of each translation unit to bedisplayed on the display screen are associated with one another.”

For example, as the display is performed as illustrated in FIG. 14, itis implemented that “the translation result of each translation unit isemphasized by either or both of the color and the transparency so thatthe translation result of each translation unit in which a higher degreeof reliability (corresponding to a score) is set in the translationprocess is more noticeable.” Further, needless to say, the example ofthe display in a case in which each translation result of eachtranslation unit is caused to be displayed in either or both of thecolor corresponding to the degree of reliability and the transparencycorresponding to the degree of reliability is not limited to the exampleillustrated in FIG. 14.

(III) Third Example of Notification Control Process: NotificationControl Process Based on Voice Information

In a case in which the visual notification is given by causing thenotification content to be displayed on the display screen of thedisplay device, the information processing apparatus according to thepresent embodiment controls a method of displaying the notificationcontent on the basis of the voice information.

The information processing apparatus according to the present embodimentcontrols a method of displaying the notification content on the basis ofthe voice information by, for example, “causing the notification contentto be displayed with a size corresponding to the sound pressure or thevolume specified from the voice information.” The information processingapparatus according to the present embodiment causes the notificationcontent to be displayed with the size corresponding to the soundpressure or the volume specified from the voice information withreference to, for example, a “table (or a database) in which the soundpressure or the sound volume, a display size for causing the divisionaltext to be displayed, and a font size are associated with one another.”

Further, in a case in which the summarized content of the speech istranslated into another language according to the translation processaccording to the present embodiment, the information processingapparatus according to the present embodiment can control the method ofdisplaying the translation result on the basis of the voice information,similarly to the case of controlling the method of displaying thenotification content.

FIG. 15 illustrates a fifth example in which the translation result isdisplayed on the display screen of the display device, and illustratesan example in which the translation result is emphatically displayed onthe basis of the voice information. In the example illustrated in FIG.15, each of “recommendation,” “sightseeing,” “route,” “tell,” and“Asakusa” corresponds to the translation results of each translationunit. Further, FIG. 15 illustrates an example in which the soundpressure or the sound volume decreases, for example, in the order of“tell,” “route,” “recommendation,” “sightseeing,” and “Asakusa.”

The information processing apparatus according to the present embodimentcauses the translation result of each translation unit (the translatedsummarized content of the speech) to be displayed with the sizecorresponding to the sound pressure or the volume specified from thevoice information, for example, as illustrated in A of FIG. 15. Theinformation processing apparatus according to the present embodimentcauses the translation result to be displayed with the sizecorresponding to the sound pressure or the volume specified from thevoice information with reference to, for example, a “table (or adatabase) in which the sound pressure or the sound volume, a displaysize for causing the translation result of each translation unit to bedisplayed, and a font size are associated with one another.”

For example, as the display is performed as illustrated in FIG. 15, itis implemented “that the display is performed with the font size and thedisplay size are increased so that the translation result with thehigher sound pressure (or volume) is more noticeable.” Further, needlessto say, the example of the display in a case in which the display methodis controlled on the basis of the voice information is not limited tothe example illustrated in FIG. 15.

(IV) Fourth Example of Notification Control Process: NotificationControl Process Based on Manipulation Performed on Display Screen

In a case in which the visual notification is given by causing thenotification content to be displayed on the display screen of thedisplay device, the information processing apparatus according to thepresent embodiment causes content being displayed on the display screento be changed on the basis of a manipulation performed on the displayscreen.

Here, as the manipulation performed on the display screen, an arbitrarymanipulation which can be performed on the display screen such as amanipulation using a manipulation input device such as a button, adirection key, a mouse, or a keyboard, a manipulation on the displayscreen (in a case in which the display device is a touch panel) may beused.

The information processing apparatus according to the present embodimentcauses the content being displayed on the display screen to be changedon the basis of the manipulation performed on the display screen, forexample, by performing either or both of the following processes (IV-1)and (IV-2).

(IV-1) First Example of Notification Control Process Based onManipulation Performed on Display Screen

The information processing apparatus according to the present embodimentcauses the content displayed on the display screen to be changed on thebasis of the manipulation performed on the display screen. As an exampleof causing the content being displayed on the display screen accordingto the present embodiment to be changed, one or more of examplesdescribed below may be used.

-   -   Change of a display position of the notification content on the        display screen (or change of a display position of the        translation result on the display screen)    -   Deletion of some of the notification content being displayed on        the display screen (or deletion of some of the translation        results being displayed on the display screen)

The information processing apparatus according to the present embodimentcan manually change, for example, the content to be presented to thecommunication partner by causing the display position of thenotification content on the display screen (or the display position ofthe translation result on the display screen) to be changed on the basisof the manipulation performed on the display screen. Further, theinformation processing apparatus according to the present embodiment canmanually delete, for example, the erroneous translation result or thelike by causing a part of the notification content being displayed onthe display screen (or a part of the translation result being displayedon the display screen) to be deleted on the basis of the manipulationperformed on the display screen.

FIGS. 16A, 16B, 16C, 16D, and 16E illustrate examples of the displayscreen in the case of causing the content being displayed on the displayscreen to be changed on the basis of the manipulation performed on thedisplay screen. Here,

FIG. 16A illustrates an example of display in a case in which thetranslation result of each translation unit according to the translationprocess is retranslated. Further, FIGS. 16B and 16C illustrate anexample of display in a case in which a part of the translation resultof each translation unit (the translated summarized content of thespeech) being displayed on the display screen are deleted. Further,FIGS. 16D and 16E illustrate an example of display in a case in whichthe display position of the translation result of each translation unit(the translated summarized content of the speech) being displayed on thedisplay screen is changed.

For example, a case in which the user desires to delete “recommendation”which is a part of translation result of each translation unit beingdisplayed on display screen will be described as an example. If the userselects “recommendation” as indicated by reference numeral O in FIG.16B, a window W for selecting whether to delete is displayed asillustrated in FIG. 16B. Further, if the user selects “Yes” in thewindow W, “recommendation” which is a part of the translation result isdeleted as illustrated in FIG. 16C. Further, needless to say, an exampleof deleting a part of the translation result of each translation unitbeing displayed on the display screen is not limited to the exampleillustrated in FIGS. 16B and 16C.

Further, for example, a case in which the user desires to change thedisplay positions of “recommendation” and “tell” in the translationresult of each translation unit being displayed on the display screenwill be described as an example. For example, if the user selects “tell”as indicated by reference numeral O1 in FIG. 16D and then designates aposition indicated by reference numeral O2 in FIG. 16E according to adrag manipulation, the display positions of “recommendation” and “tell”are switched as illustrated in FIG. 16E. Further, needless to say, theexample of changing the display position of the translation result ofeach translation unit being displayed on the display screen is notlimited to the example illustrated in FIGS. 16D and 16E.

(IV-2) Second Example of Notification Control Process Based onManipulation Performed on Display Screen

In a case in which the summarized content of the speech (or thetranslation result) is caused to be displayed on the display screen ofthe display device as the notification content, the summarized contentof the speech (or the translation result) may be unable to be displayedon a single screen. In a case in which the summarized content of thespeech (or the translation result) is unable to be displayed on a singlescreen as described above, the information processing apparatusaccording to the present embodiment causes a part of the notificationcontent to be displayed on the display screen.

Further, in a case in which a part of the notification content is beingdisplayed on the display screen, the information processing apparatusaccording to the present embodiment causes the content being displayedon the display screen to be changed on the basis of the manipulationperformed on the display screen. The information processing apparatusaccording to the present embodiment causes the content being displayedon the display screen to be changed by, for example, causing thenotification content being displayed on the display screen from the partto another part.

FIGS. 17 and 18 illustrate examples of the display screen in the case ofcausing the translation result of each translation unit obtainedaccording to the translation process (the translated summarized contentof the speech) to be changed on the basis of the manipulation performedon display screen. Here, FIG. 17 illustrates an example of a displayscreen in which it is possible to cause the content being displayed onthe display screen to be changed through a slider type UI as illustratedin A of FIG. 17. Further, FIG. 18 illustrates an example of a displayscreen in which it is possible to cause the content being displayed onthe display screen to be changed through a revolver type UI in which adisplay is changed while rotating in the depth direction of the displayscreen.

For example, a case in which the user desires to change content beingdisplayed on the display screen in a state in which the displayillustrated in FIG. 17 is performed will be described as an example. Forexample, the user causes the translation result being displayed on thedisplay screen to be changed from a part to another part by manipulatingthe slider type UI according to a touch manipulation or the like oftouching an arbitrary part of the slider illustrated in A of FIG. 17.

Further, for example, a case in which the user changes the content beingdisplayed on the display screen in a state in which the displayillustrated in FIG. 18 is performed will be described as an example. Forexample, the user causes the translation result being displayed on thedisplay screen to be changed from a part to another part by manipulatingthe revolver type UI by performing a flick manipulation indicated byreference numeral O1 in FIG. 18 or the like.

Further, needless to say, the example of changing the translation resultbeing displayed on the display screen is not limited to the exampleillustrated in FIGS. 17 and 18.

(V) Fifth Example of Notification Control Process: Notification ControlProcess Based on Voice-Based Manipulation

The information processing apparatus according to the present embodimentmay cause an auditory notification of the translation result to be giventhrough a voice from the voice output device on the basis of avoice-based manipulation.

FIGS. 19A and 19B illustrate an example of a case in which an auditorynotification of the translation result is given on the basis of thevoice-based manipulation. FIGS. 19A and 19B illustrate an example of acase in which content whose notification is given to the communicationpartner is selected from the translation result of each translation unitobtained according to the translation process on the basis of thevoice-based manipulation.

For example, in a case in which the translation result for eachtranslation unit obtained according to the translation process includes“recommendation,” “sightseeing” “route,” and “tell,” the informationprocessing apparatus according to the present embodiment causes anotification of a retranslation result to be given through a voice asindicated by reference numeral “I1” in FIG. 19A as illustrated in

FIG. 19A. At this time, as illustrated in FIG. 19A, the informationprocessing apparatus according to the present embodiment may insert asound feedback indicated by reference numeral “S” in FIG. 19A in thedelimiter of the divisional text.

In a case in which a voice-based manipulation as indicated by referencenumeral “O” in FIG. 19B is detected after a notification of theretranslation result is given through a voice, the informationprocessing apparatus according to the present embodiment causes a voiceindicating the translation result corresponding to the voice-basedselection manipulation to be output from the voice output device asindicated by reference numeral “I2” in FIG. 19B. Here, FIG. 19Billustrates an example of the voice-based selection manipulation fordesignating content whose notification is given to the communicationpartner using a number. Further, needless to say, the example of thevoice-based selection manipulation according to the present embodimentis not limited to the above example.

FIGS. 20A and 20B illustrate another example of a case in which anauditory notification of the translation result is given on the basis ofthe voice-based manipulation. FIGS. 20A and 20B illustrate an example ofa case in which content whose notification is given to the communicationpartner is excluded from the translation result of each translation unitobtained according to the translation process on the basis of thevoice-based manipulation.

For example, in a case in which the translation result for eachtranslation unit obtained according to the translation process includes“recommendation,” “sightseeing” “route,” and “tell,” the informationprocessing apparatus according to the present embodiment causes anotification of a retranslation result to be given through a voice asindicated by reference numeral “I1” in FIG. 20A as illustrated in FIG.20A. Further, similarly to FIG. 19A, the information processingapparatus according to the present embodiment may insert a soundfeedback in the delimiter of the divisional text.

In a case in which a voice-based exclusion manipulation indicated byreference numeral “O” in of FIG. 20B is detected after a notification ofthe retranslation result is given through a voice, the informationprocessing apparatus according to the present embodiment causes a voiceindicating the translation result corresponding to the voice-basedselection manipulation to be output from the voice output device asindicated by reference numeral “I2” in FIG. 20B. Here, FIG. 20Billustrates an example of the voice-based exclusion manipulation fordesignating content whose notification need not be given to thecommunication partner using a number. Further, needless to say, theexample of the voice-based exclusion manipulation according to thepresent embodiment is not limited to the above example.

Further, needless to say, the example of the voice-based manipulationand the example of the notification based on the voice-basedmanipulation is not limited to the examples illustrated in FIGS. 19A19B, 20A, and 20B.

(VI) Sixth Example of Notification Control Process: Notification ControlProcess in Case in which Notification Order is Dynamically Controlled

The information processing apparatus according to the present embodimentcan also dynamically control the notification order of the notificationcontent.

The information processing apparatus according to the present embodimentcontrols the notification order of the notification content, forexample, on the basis of at least one of information corresponding to afirst user and information corresponding to a second user. Theinformation corresponding to the first user includes, for example, atleast one of information related to the first user, information relatedto an application, and information related to a device. Further, theinformation corresponding to the second user includes at least one ofinformation related to the second user, information related to anapplication, and information related to a device.

The information related to the first user indicates, for example, eitheror both of a situation in which the first user is placed and a state ofthe first user. Further, the information related to the second userindicates, for example, either or both of a situation in which thesecond user is placed and a state of the second user. Further, theinformation related to the application indicates, for example, theexecution state of the application as described above. Further, theinformation related to the device indicates, for example, either or bothof the type of device and the state of the device as described above.

The situation in which the user (the first user or the second user) isplaced is estimated by an arbitrary method capable of estimating thesituation in which the user is placed such as, for example, a method ofestimating on the basis of noise around the user (for example, a soundother than the voice based on the speech) detected from the voiceinformation or a method of estimating the situation in which the user isplaced on the basis of the position indicated by the positioninformation. The process of estimating the situation in which the useris placed may be performed by the information processing apparatusaccording to the present embodiment or may be performed in the externalapparatus of the information processing apparatus according to thepresent embodiment.

Further, as described above, the state of the user is estimatedaccording to an arbitrary behavior estimation process or an arbitraryemotion estimation process using one or more of biometric information ofthe user, a detection result of a motion sensor, a captured imagecaptured by an imaging device, and the like.

FIGS. 21A, 21B, and 21C illustrate an example of display in a case inwhich the notification order is dynamically controlled. FIG. 21Aillustrates an example in which the translation result of eachtranslation unit obtained according to the translation process (thetranslated summarized content of the speech) is displayed on the basisof the state of the user. Further, FIG. 21B illustrates an example inwhich the translation result of each translation unit obtained accordingto the translation process is displayed on the basis of the executionstate of the application. Further, FIG. 21C illustrates an example inwhich the translation result of each translation unit obtained accordingto the translation process is displayed on the basis of the situation inwhich the user is placed.

FIG. 21A illustrates an example of display based on the state of theuser in a case in which the translation result of each translation unitincludes “recommendation,” “sightseeing,” “route,” and “tell.”

For example, in a case in which the state of the user is recognized as“impatience” on the basis of the biometric information, the detectionresult of the motion sensor, or the like, the information processingapparatus according to the present embodiment causes a verb to bedisplayed with priority by displaying a verb to be displayed on theleftmost side of the display screen as illustrated in FIG. 21A. Theinformation processing apparatus according to the present embodimentspecifies the notification order with reference to, for example, a“table (or a database)” in which the state of the user is associatedwith information indicating the display order.

FIG. 21B illustrates an example of display based on the execution stateof the application in a case in which the translation result of eachtranslation unit includes “Hokkaido,” “production area,” “delicious,”and “fish.”

For example, in a case in which the type of application being executedis recognized as a “meal browser” in an apparatus associated with theuser such as the smartphone possessed by the user, the informationprocessing apparatus according to the present embodiment causes anadjective to be displayed with priority by displaying an adjective to bedisplayed on the leftmost side of the display screen as illustrated inFIG. 21B. The information processing apparatus according to the presentembodiment specifies the notification order with reference to, forexample, a “table (or a database)” in which the type of application isassociated with the information indicating the display order.

FIG. 21C illustrates an example of display based on the situation inwhich the user is placed in a case in which the translation result ofeach translation unit includes “in hurry,” “Shibuya,” “gather,” and“have no time”.

For example, in a case in which the noise detected from the voiceinformation (for example, a sound other than the voice based on thespeech) is larger than a set threshold value, the information processingapparatus according to the present embodiment recognizes that the useris in a noisy situation. Then, the information processing apparatusaccording to the present embodiment causes a noun (or a proper noun) tobe displayed with priority by displaying a noun (or a proper noun) to bedisplayed on the leftmost side of the display screen as illustrated inFIG. 21C. The information processing apparatus according to the presentembodiment specifies the notification order with reference to, forexample, a “table (or a database)” in which an environment in which useris placed is associated with the information indicating the displayorder.

Further, the example of dynamically controlling the notification orderis not limited to the example illustrated in FIGS. 21A, 21B, and 21C.

For example, in a case in which the notification order is dynamicallycontrolled on the basis of two or more of the situation in which theuser is placed, the state of the user, and the execution state of theapplication (an example of a case in which the notification order isdynamically changed on the basis of a plurality of pieces ofinformation), the information processing apparatus according to thepresent embodiment specifies the notification order on the basis of apriority (or a priority order) set in each of the situation in which theuser is placed, the state of the user, and the execution state of theapplication. The information processing apparatus according to thepresent embodiment causes a notification of the notification contentcorresponding to an index having a high priority (or priority order) tobe given with priority.

Further, FIGS. 21A, 21B, and 21C illustrate the example of thenotification based on the visual method, but as described above, theinformation processing apparatus according to the present embodiment canalso give a notification based on the auditory method.

Further, the information processing apparatus according to the presentembodiment can dynamically control the notification order on the basisof each piece of information related to the device as described above.As an example of dynamically controlling the notification order on thebasis of the information related to the device, for example, thenotification order may be dynamically controlled in accordance with aprocessing load of a processor.

(VII) Sixth Example of Notification Control Process: NotificationControl Process in a Case in which Notification Content is DynamicallyControlled

The information processing apparatus according to the present embodimentcan dynamically control an information amount of the notificationcontent.

The information processing apparatus according to the present embodimentdynamically controls an information amount of the notification contenton the basis of one or more of, for example, summary information, theinformation corresponding to the first user, the informationcorresponding to the second user, and the voice information. An exampleof dynamically changing the information amount, for example, thefollowing examples (VII-1) to (VII-5) may be used. Further, needless tosay, the example of dynamically changing the information amount is notlimited to the following examples (VII-1) to (VII-5).

(VII-1) Example of Dynamically Changing Notification Content Based onSummary Information

-   -   For example, in a case in which a demonstrative such as “that”        or “it” is included in the summarized content of the speech        indicated by the summary information, the information processing        apparatus according to the present embodiment causes a        notification of the demonstrative (or the translation result of        the demonstrative) not to be given.    -   For example, in a case in which a word corresponding to a        greeting is included in the summarized content of the speech        indicated by the summary information, the information processing        apparatus according to the present embodiment causes a        notification of the word corresponding to the greeting (or the        translation result of the word corresponding to the greeting)        not to be given.        (VII-2) Example of Dynamically Changing Notification Content        Based on Information Corresponding to First User

For example, in a case in which a facial expression of the first user isdetermined to be a smile, the information processing apparatus accordingto the present embodiment reduces the information amount in a case inwhich a notification of the notification content is given. For example,in a case in which the line of sight of the first user is determined toface upward (an example of a case in which it is determined to be closeto a monologue), the information processing apparatus according to thepresent embodiment causes a notification of the notification content notto be given. For example, in a case in which a gesture corresponding toa demonstrative such as “that,” “it,” or “this” (for example, a pointinggesture or the like) is detected, the information processing apparatusaccording to the present embodiment causes a notification of thenotification content not to be given. For example, in a case in whichthe first user is determined to be placed in a situation in which thenoise is large, the information processing apparatus according to thepresent embodiment causes notifications of all the notification contentto be given.

(VII-3) Example of Dynamically Changing Notification Content Based onInformation Corresponding to Second User

-   -   For example, in a case in which the facial expression of the        second user is determined to be a smile, the information        processing apparatus according to the present embodiment reduces        the information amount when a notification of the notification        content is given.    -   In a case in which the second user is the communication partner,        for example, when it is determined that there is a possibility        that the second user does not understand the content of the        speech (for example, when it is determined that the line of        sight of the second user does not face the first user), the        information processing apparatus according to the present        embodiment increases the information amount when a notification        of the notification content is given.    -   In a case in which the second user is the communication partner,        for example, when it is determined that the second user is        yawning (for example, when it is determined that the second user        feels bored or the like), the information processing apparatus        according to the present embodiment decreases the information        amount when a notification of the notification content is given.    -   In a case in which the second user is the communication partner,        for example, when it is determined the second user nods or        agrees enthusiastically, the information processing apparatus        according to the present embodiment increases the information        amount when a notification of the notification content is given.    -   In a case in which the second user is the communication partner,        for example, when it is determined that the size of the pupil of        the second user is larger than a predetermined size or when it        is determined that the size of the pupil is equal to or larger        than a predetermined size (an example of a case in which the        user is determined to be interested in), the information        processing apparatus according to the present embodiment        increases the information amount when a notification of the        notification content is given.    -   In a case in which the second user is the communication partner,        for example, when it is determined that there is a possibility        that the second user does not understand the content of the        speech (for example, when it is determined that the hand of the        second user is not moving or the like), the information        processing apparatus according to the present embodiment        increases the information amount when a notification of the        notification content is given.    -   In a case in which the second user is the communication partner,        for example, when it is determined that an inclination state of        the body of the second user is leaning forward (an example in a        case in which the user is determined to be interested in), the        information processing apparatus according to the present        embodiment increases the information amount when a notification        of the notification content is given.

For example, in a case in which it is determined that the second user isplaced in a situation in which the noise is large, the informationprocessing apparatus according to the present embodiment causesnotifications of all the notification content to be given.

(VII-4) Example of dynamically changing notification content based onvoice information

-   -   For example, in a case in which the volume of the speech        detected from the voice information is larger than a        predetermined threshold value or in a case in which the volume        of the speech is equal to or more than a predetermined threshold        value, the information processing apparatus according to the        present embodiment causes a notification of the notification        content not to be given. For example, in a case in which the        volume of the speech detected from the voice information is        larger than a predetermined threshold value or in a case in        which the volume of the speech is equal to or more than a        predetermined threshold value, the information processing        apparatus according to the present embodiment causes a        notification of a part or all of the notification content to be        given.        (VII-5) Example of Dynamically Changing Notification Content        Based on Combination of Plural Information

In a case in which the first user and the second user are different fromeach other, for example, when it is determined that the line of sight ofthe first user matches the line of sight of the second user, theinformation processing apparatus according to the present embodimentincreases the information amount when a notification of the notificationcontent is given (an example of dynamic changing the notificationcontent on the basis of the information corresponding to the first userand the information corresponding to the second user).

Specific example of process according to information processing methodaccording to present embodiment

Next, a specific example of the process according to the informationprocessing method of the present embodiment described above will bedescribed. An example of the process in the use case described withreference to FIGS. 1, 2, 3A, 3B, 3C, 4A, 4B, and 5 will be describedbelow as a specific example of the process according to the informationprocessing method of the present embodiment.

FIGS. 22 to 33 are flowcharts illustrating an example of the processaccording to the information processing method of the presentembodiment. An example of the process according to the informationprocessing method of the present embodiment will be described below withreference to FIGS. 22 and 33.

The information processing apparatus according to the present embodimentsets the weight related to the summary (hereinafter also referred to asa “weight related to a summarization function” or referred to simply asa “weight”) (S100) (presetting). The information processing apparatusaccording to the present embodiment sets the weight related to thesummary by deciding the weight related to the summary and holding theweight related to the summary in a recording medium such as a storageunit (to be described later). As the process of step S100, for example,the process illustrated in FIG. 23 may be performed.

Referring to FIG. 23, the information processing apparatus according tothe present embodiment acquires data indicating schedule content fromthe scheduling application (S200).

The information processing apparatus according to the present embodimentdecides the type of weight related to the summary on the basis of abehavior recognized from the data indicating the acquired schedulecontent and a table for specifying the type of weight related to thesummary illustrated in FIG. 8 (hereinafter referred to as “behaviorinformation summary weight table”) (S202).

Then, the information processing apparatus according to the presentembodiment decides the weight related to the summary on the basis of thetype of weight related to the summary decided in step S202 and a tablefor specifying the weight related to the summary illustrated in FIG. 6(hereinafter also referred to as a “summary table”) (S204).

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 23 as the processof step S100 in FIG. 22. Further, needless to say, the process of stepS100 in FIG. 22 is not limited to the process illustrated in FIG. 23.

An example of the process according to the information processing methodof the present embodiment will be described with reference to FIG. 22again. The information processing apparatus according to the presentembodiment enables a voice input (S102), for example, when anapplication related to the voice input is activated.

The information processing apparatus according to the present embodimentdetermines whether or not the voice information is acquired (S104). In acase in which it is determined in step S104 that the voice informationis not acquired, the information processing apparatus according to thepresent embodiment does not proceed with the process of step S106 andsubsequent steps, for example, until it is determined that the voiceinformation is acquired.

Further, in a case in which it is determined in step S104 that the voiceinformation is acquired, the information processing apparatus accordingto the present embodiment analyzes the voice information (S106). Theinformation processing apparatus according to the present embodimentanalyzes the voice information and obtains, for example, the soundpressure, the pitch, the average frequency band, and the like. Further,the information processing apparatus according to the present embodimentholds the voice information in a recording medium such as a storage unit(to be described later) (S108).

The information processing apparatus according to the present embodimentsets the weight related to the summary on the basis of the voiceinformation or the like (S110). As the process of step S110, forexample, the process illustrated in FIG. 24 may be performed.

Referring to FIG. 24, the information processing apparatus according tothe present embodiment sets the weight related to the summary on thebasis of the average frequency of the voice indicated by the voiceinformation (hereinafter also referred to as an “input voice”) (S300).As the process of step S300, for example, the process illustrated inFIG. 25 may be performed.

Further, FIG. 24 illustrates the example in which the process of stepS302 is performed after the process of step S300, but the process ofstep S110 of FIG. 22 is not limited to the process illustrated in FIG.24. For example, since the process of step S300 and the process of stepS302 are independent processes, the information processing apparatusaccording to the present embodiment can perform the process of step S304after the process of step S302 or can perform the process of step S300and the process of step S302 in parallel.

Referring to FIG. 25, the information processing apparatus according tothe present embodiment determines whether or not the average frequencyband of the voice is 300 [Hz] to 550 [Hz] (S400).

In a case in which it is determined in step S400 that the averagefrequency band of the voice is 300 [Hz] to 550 [Hz], the informationprocessing apparatus according to the present embodiment decides “male”as the type of weight related to the summary (S402).

Further, in a case in which it is determined in step S400 that theaverage frequency band of the voice is not 300 [Hz] to 550 [Hz], theinformation processing apparatus according to the present embodimentdetermines whether or not the average frequency band of the voice is,400 [Hz] to 700 [Hz] (S404).

In a case in which it is determined in step S404 that the averagefrequency band of the voice is 400 [Hz] to 700 [Hz], the informationprocessing apparatus according to the present embodiment decides“female” as the type of weight related to the summary (S406).

Further, in a case in which it is determined in step S404 that theaverage frequency band of the voice is not 400 [Hz] to 700 [Hz], theinformation processing apparatus according to the present embodimentdoes not decide the weight related to the summary.

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 25 as the processof step S300 in FIG. 24. Further, needless to say, the process of stepS300 in FIG. 24 is not limited to the process illustrated in FIG. 25.

An example of the process of step S110 in FIG. 22 will be described withreference to FIG. 24 again. The information processing apparatusaccording to the present embodiment sets the weight related to thesummary on the basis of the sound pressure of the voice indicated by thevoice information (S302). As the process of step S302, for example, theprocess illustrated in FIG. 26 may be performed.

Referring to FIG. 26, the information processing apparatus according tothe present embodiment decides a threshold value related to the soundpressure on the basis of a distance between the user of the speaker andthe communication partner (S500). As the process of step S500, forexample, the process illustrated in FIG. 27 may be performed.

Referring to FIG. 27, the information processing apparatus according tothe present embodiment acquires a distance D from the currentcommunication partner according to image recognition based on thecaptured image captured by the imaging device (S600).

The information processing apparatus according to the present embodimentperforms, for example, an operation of the following Formula 2 (S602).

[Math.  2]                                        $\begin{matrix}{R = {\left( {- 1} \right) \cdot \frac{\left( {D - 1} \right)}{0.5}}} & \left( {{Formula}\mspace{14mu} 2} \right)\end{matrix}$

Then, the information processing apparatus according to the presentembodiment performs, for example, an operation of the following Formula3, and decides a threshold value related to the sound pressure byadjusting a threshold value VPWR_thresh_upper related to the soundpressure and a threshold value VPWR_thresh_lower related to the soundpressure (S604).

[Math.  3]                                        $\begin{matrix}\left\{ \begin{matrix}{{{VPWR}_{—}{thresh}_{—}{upper}\text{+}} = {R \cdot 6}} \\{{{VPWR}_{—}{thresh}_{—}{lower}\text{+}} = {R \cdot 6}}\end{matrix} \right. & \left( {{Formula}\mspace{14mu} 3} \right)\end{matrix}$

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 27 as the processof step S500 in FIG. 26. Further, needless to say, the process of stepS500 in FIG. 26 is not limited to the process illustrated in FIG. 27.

An example of the process of step S302 of FIG. 24 will be described withreference to FIG. 26 again. The information processing apparatusaccording to the present embodiment determines whether or not the soundpressure of the voice indicated by the voice information is equal to orlarger than the threshold value VPWR_thresh_upper related to the soundpressure (S502).

In a case in which it is determined in step S502 that the sound pressureof the voice indicated by the voice information is equal to or largerthan the threshold value VPWR_thresh_upper related to the soundpressure, the information processing apparatus according to the presentembodiment decides “anger” and “joy” as the type of weight related tothe summary (S504).

Further, in a case in which it is determined in step S502 that the soundpressure of the voice indicated by the voice information is neitherequal to nor larger than the threshold value VPWR_thresh_upper relatedto the sound pressure, the information processing apparatus according tothe present embodiment determines whether or not the sound pressure ofthe voice indicated by the voice information is equal to or less thanthe threshold value VPWR_thresh_lower related to the sound pressure(S506).

In a case in which it is determined in step S506 that the sound pressureof the voice indicated by the voice information is equal to or less thanthe threshold value VPWR_thresh_lower related to the sound pressure, theinformation processing apparatus according to the present embodimentdecides “sadness,” “discomfort,” “distress,” and “anxiety” as the typeof weight related to the summary (S508).

Further, in a case in which it is determined in step S506 that the soundpressure of the voice indicated by the voice information is neitherequal to nor less than the threshold value VPWR_thresh_lower related tothe sound pressure, the information processing apparatus according tothe present embodiment does not decide the weight related to thesummary.

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 26 as the processof step S302 in FIG. 24. Further, needless to say, the process of stepS302 in FIG. 24 is not limited to the process illustrated in FIG. 26.

An example of the process of step S110 of FIG. 22 will be described withreference to FIG. 24 again. The information processing apparatusaccording to the present embodiment analyzes, for example, the voiceinformation and holds the number of moras and the position of the accent(S304). Further, the process of step S304 may be performed in theprocess of step S106 of FIG. 22.

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 24 as the processof step S110 in FIG. 22. Further, needless to say, the process of stepS110 in FIG. 22 is not limited to the process illustrated in FIG. 24.

An example of the process according to the information processing methodof the present embodiment will be described with reference to FIG. 22again. The information processing apparatus according to the presentembodiment performs the voice recognition on the voice information(S112). As the process of step S112 is performed, the voice textinformation is acquired.

If the process of step S112 is performed, the information processingapparatus according to the present embodiment sets the weight related tothe summary on the basis of the voice recognition result or the like(S114). As the process of step S114, for example, the processillustrated in FIG. 28 may be performed.

Referring to FIG. 28, the information processing apparatus according tothe present embodiment sets the weight related to the summary on thebasis of the language of the character string indicated by the voicetext information (S700). As the process of step S700, for example, theprocess illustrated in FIG. 29 may be performed.

Further, FIG. 28 illustrates the example in which the processes of stepsS704 to S710 are performed after the processes of steps S700 and S702,but the process of step S114 of FIG. 22 is not limited to the processillustrated in FIG. 28. For example, since the processes of steps S700and S702 and the processes of steps S704 to S710 are independentprocesses, the information processing apparatus according to the presentembodiment can perform the processes of steps S700 and S702 after theprocesses of steps S704 to S710 or can perform the processes of stepsS700 and S702 and the processes of steps S704 to S710 in parallel.

Referring to FIG. 29, the information processing apparatus according tothe present embodiment estimates the language of the character stringindicated by the voice text information (S800). The informationprocessing apparatus according to the present embodiment estimates thelanguage according to a process according to an arbitrary method capableof estimating the language from the character string such as estimationby matching with a language dictionary

If the language is estimated in step S800, the information processingapparatus according to the present embodiment determines whether or notthe estimated language is Japanese (S802).

In a case in which it is determined in step S802 that the estimatedlanguage is Japanese, the information processing apparatus according tothe present embodiment decides the weight related to the summary so thatthe weight of “a verb of Japanese” is increased (S804).

Further, in a case in which it is determined in step S802 that theestimated language is not Japanese, the information processing apparatusaccording to the present embodiment determines whether or not theestimated language is English (S806).

In a case in which it is determined in step S806 that the estimatedlanguage is English, the information processing apparatus according tothe present embodiment decides the weight related to the summary so thatthe weights of “a noun and a verb of English” are increased (S808).

Further, in a case in which it is determined in step S806 that theestimated language is not English, the information processing apparatusaccording to the present embodiment does not decide the weight relatedto the summary.

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 29 as the processof step S700 in FIG. 28. Further, needless to say, the process of stepS700 in FIG. 28 is not limited to the process illustrated in FIG. 29.

An example of the process of step S114 of FIG. 22 will be described withreference to FIG. 28 again. The information processing apparatusaccording to the present embodiment analyzes, for example, the voiceinformation and holds the number of moras and the position of the accent(S702). Further, the process of step S702 may be performed in theprocess of step S106 of FIG. 22.

The information processing apparatus according to the present embodimentdivides the character string indicated by the voice text information(hereinafter also referred to as a “voice text result”) into units ofmorphemes according to a natural language process and links analysisresults of corresponding voice information (S704).

The information processing apparatus according to the present embodimentestimates an emotion on the basis of the analysis results of the voiceinformation linked in units of morphemes in step S704 (S706). Theinformation processing apparatus according to the present embodiment canestimate emotion using an arbitrary method capable of estimating theemotion using the analysis result of the voice information such as amethod of using a table in which the analysis result of the voiceinformation is associated with the emotion.

Further, the information processing apparatus according to the presentembodiment decides the strength of the weight related to the summary(the strength of the weight related to the emotion) on the basis of theanalysis results of the voice information linked in units of morphemesin step S704 (S708). The information processing apparatus according tothe present embodiment decides the strength of the weight related to thesummary on the basis of, for example, a change rate of a basicfrequency, a change rate of a sound, a change rate of a speaking timeamong the analysis results of the voice information. The informationprocessing apparatus according to the present embodiment decides thestrength of the weight related to the summary using an arbitrary methodcapable of deciding the strength of the weight related to the summaryusing the analysis result of the voice information such as a method ofusing a table in which the analysis result of the voice information isassociated with the strength of the weight related to the summary.

The information processing apparatus according to the present embodimentdecides the weight related to the summary on the basis of the emotionestimated in step S706 (S710). Further, the information processingapparatus according to the present embodiment may adjust the weightrelated to the summary decided on the basis of the estimated emotion inaccordance with the strength of the weight related to the summarydecided in step S708.

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 28 as the processof step S114 in FIG. 22. Further, needless to say, the process of stepS114 in FIG. 22 is not limited to the process illustrated in FIG. 28.

An example of the process according to the information processing methodof the present embodiment will be described will be described withreference to FIG. 22 again. The information processing apparatusaccording to the present embodiment performs the summarization processon the basis of the weight related to the summary decided in each ofsteps S100, S110, and S114 (S116).

If the process of step S116 is completed, the information processingapparatus according to the present embodiment determines whether or notthe translation process is performed (S118).

In a case in which it is determined in step S118 that the translationprocess is not to be performed, the information processing apparatusaccording to the present embodiment causes a notification of thesummarization result to be given according to the notification controlprocess (S120).

Further, in a case in which it is determined in step S118 that thetranslation process is performed, the information processing apparatusaccording to the present embodiment performs the translation process onthe summarization result and causes a notification of the translationresult to be given according to the notification control process (S122).As the process of step S122, for example, the process illustrated inFIG. 30 may be performed.

Referring to FIG. 30, the information processing apparatus according tothe present embodiment performs the morphological analysis byperforming, for example, the natural language process on thesummarization result (S900).

The information processing apparatus according to the present embodimentgenerates the divisional text obtained by combining the main parts ofspeech (a noun, a verb, an adjective, and an adverb) with othermorphemes until there is no unprocessed summarization result (S902).

The information processing apparatus according to the present embodimentdetermines whether or not the language of the summarization result isEnglish (S904).

In a case in which it is determined in step S904 that the language ofthe summarization result is not English, the information processingapparatus according to the present embodiment performs a process of stepS908 to be described later.

Further, in a case in which it is determined in step S904 that thelanguage of the summarization result is English, the informationprocessing apparatus according to the present embodiment uses wordscorresponding to 5W1H as the divisional text (S906).

In a case in which it is determined in step S904 that the language ofthe summarization result is not English or if the process of step S906is performed, the information processing apparatus according to thepresent embodiment performs the translation process on each divisionaltext, links the translation result with information of the original partof speech before translation, and holds the resulting information(S908).

The information processing apparatus according to the present embodimentdetermines whether or not the language of the divisional translationtext (an example of the translation result) is English (S910).

In a case in which it is determined in step S910 that the language ofthe divisional translation text is English, the information processingapparatus according to the present embodiment decides the notificationorder in English (S912). As the process of step S912, for example, theprocess illustrated in FIG. 31 may be performed.

Referring to FIG. 31, the information processing apparatus according tothe present embodiment determines whether or not there is a divisionaltranslation text to be processed (S1000). Here, an unprocessedtranslation result among the translation results of each translationunit corresponds to the divisional translation text to be processed instep S1000. For example, the information processing apparatus accordingto the present embodiment determines that there is a divisionaltranslation text to be processed in a case in which there is anunprocessed translation result, and determines that there is nodivisional translation text to be processed in a case in which there isno unprocessed translation result.

In a case in which it is determined in step S1000 that there is adivisional translation text to be processed, the information processingapparatus according to the present embodiment acquires the divisionaltranslation text to be processed next (S1002).

The information processing apparatus according to the present embodimentdetermines whether or not the divisional translation text to beprocessed includes a noun (S1004).

In a case in which it is determined in step S1004 that the divisionaltranslation text to be processed includes a noun, the informationprocessing apparatus according to the present embodiment sets thepriority to a maximum value “5” (S1006). Then, the informationprocessing apparatus according to the present embodiment repeats theprocess starting from step S1000.

Further, in a case in which it is determined in step S1004 that thedivisional translation text to be processed does not include a noun, theinformation processing apparatus according to the present embodimentdetermines whether or not the divisional translation text to beprocessed includes a verb (S1008).

In a case in which it is determined in step S1008 that the divisionaltranslation text to be processed includes a verb, the informationprocessing apparatus according to the present embodiment sets thepriority to “4” (S1010). Then, the information processing apparatusaccording to the present embodiment repeats the process starting fromstep S1000.

Further, in a case in which it is determined in step S1008 that thedivisional translation text to be processed does not include a verb, theinformation processing apparatus according to the present embodimentdetermines whether or not the divisional translation text to beprocessed includes an adjective (S1012).

In a case in which it is determined in step S1012 that the divisionaltranslation text to be processed includes an adjective, the informationprocessing apparatus according to the present embodiment sets thepriority to “3” (S1014). Then, the information processing apparatusaccording to the present embodiment repeats the process starting fromstep S1000.

Further, in a case in which it is determined in step S1012 that thedivisional translation text to be processed does not include anadjective, the information processing apparatus according to the presentembodiment determines whether or not the divisional translation text tobe processed includes an adverb (S1016).

In a case in which it is determined in step S1016 that the divisionaltranslation text to be processed includes an adverb, the informationprocessing apparatus according to the present embodiment sets thepriority to “2” (S1018). Then, the information processing apparatusaccording to the present embodiment repeats the process starting fromstep S1000.

Further, in a case in which it is determined in step S1016 that thedivisional translation text to be processed does not include an adverb,the information processing apparatus according to the present embodimentsets the priority to a minimum value “1” (S1020). Then, the informationprocessing apparatus according to the present embodiment repeats theprocess starting from step S1000.

In a case in which it is determined in step S1000 that there is not adivisional translation text to be processed, the information processingapparatus according to the present embodiment sorts the notificationorder in accordance with the set priority (S1022).

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 31 as the processof step S912 in FIG. 30. Further, needless to say, the process of stepS912 in FIG. 30 is not limited to the process illustrated in FIG. 31.

An example of the process of step S122 of FIG. 22 will be described withreference to FIG. 30 again. In a case in which it is determined in stepS910 that the language of the divisional translation text is notEnglish, the information processing apparatus according to the presentembodiment decides the notification order in Japanese (S914). As theprocess of step S914, for example, the process illustrated in FIG. 32may be performed.

Referring to FIG. 32, the information processing apparatus according tothe present embodiment determines whether or not there is a divisionaltranslation text to be processed similarly to step S1100 in FIG. 31(S1100). Here, an unprocessed translation result among the translationresults of each translation unit corresponds to the divisionaltranslation text to be processed in step S1100.

In a case in which it is determined in step S1100 that there is adivisional translation text to be processed, the information processingapparatus according to the present embodiment acquires the divisionaltranslation text to be processed next (S1102).

The information processing apparatus according to the present embodimentdetermines whether or not the divisional translation text to beprocessed includes a verb (S1104).

In a case in which it is determined in step S1104 that the divisionaltranslation text to be processed includes a verb, the informationprocessing apparatus according to the present embodiment sets thepriority to a maximum value “5” (S1106). Then, the informationprocessing apparatus according to the present embodiment repeats theprocess starting from step S1100.

Further, in a case in which it is determined in step S1104 that thedivisional translation text to be processed does not include a verb, theinformation processing apparatus according to the present embodimentdetermines whether or not the divisional translation text to beprocessed includes a noun (S1108).

In a case in which it is determined in step S1108 that the divisionaltranslation text to be processed includes a noun, the informationprocessing apparatus according to the present embodiment sets thepriority to “4” (S1110). Then, the information processing apparatusaccording to the present embodiment repeats the process starting fromstep S1100.

Further, in a case in which it is determined in step S1108 that thedivisional translation text to be processed does not include a noun, theinformation processing apparatus according to the present embodimentdetermines whether or not the divisional translation text to beprocessed includes an adjective (S1112).

In a case in which it is determined in step S1112 that the divisionaltranslation text to be processed includes an adjective, the informationprocessing apparatus according to the present embodiment sets thepriority to “3” (S1114). Then, the information processing apparatusaccording to the present embodiment repeats the process starting fromstep S1100.

Further, in a case in which it is determined in step S1112 that thedivisional translation text to be processed does not include anadjective, the information processing apparatus according to the presentembodiment determines whether or not the divisional translation text tobe processed includes an adverb (S1116).

In a case in which it is determined in step S1116 that the divisionaltranslation text to be processed includes an adverb, the informationprocessing apparatus according to the present embodiment sets thepriority to “2” (S1118). Then, the information processing apparatusaccording to the present embodiment repeats the process starting fromstep S1100.

Further, in a case in which it is determined in step S1116 that thedivisional translation text to be processed does not include an adverb,the information processing apparatus according to the present embodimentsets the priority to a minimum value “1” (S1120). Then, the informationprocessing apparatus according to the present embodiment repeats theprocess starting from step S1100.

In a case in which it is determined in step S1100 that there is not adivisional translation text to be processed, the information processingapparatus according to the present embodiment sorts the notificationorder in accordance with the set priority (S1122).

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 32 as the processof step S914 in FIG. 30. Further, needless to say, the process of stepS914 in FIG. 30 is not limited to the process illustrated in FIG. 32.

An example of the process of step S122 of FIG. 22 will be described withreference to FIG. 30 again. If the process of step S912 or the processof step S914 is completed, the information processing apparatusaccording to the present embodiment causes a notification of thedivisional translation text in which the notification order is decidedto be given according to the notification control process (S916). As theprocess of step S916, for example, the process illustrated in FIG. 33may be performed.

Referring to FIG. 33, the information processing apparatus according tothe present embodiment determines whether or not there is a divisionaltranslation text to be processed similarly to step S1000 in FIG. 31(S1200). Here, an unprocessed translation result among the translationresults of each translation unit corresponds to the divisionaltranslation text to be processed in step S1200.

In a case in which it is determined in step S1200 that there is adivisional translation text to be processed, the information processingapparatus according to the present embodiment acquires the divisionaltranslation text to be processed next (S1202).

The information processing apparatus according to the present embodimentacquires the sound pressure from the voice information corresponding tothe divisional translation text to be processed, increases the soundpressure of the divisional translation text to be processed, and causesthe resulting information to be output (S1204).

The information processing apparatus according to the present embodimentdetermines whether or not the divisional translation text output in stepS1204 is the last divisional translation text (S1206). For example, theinformation processing apparatus according to the present embodimentdetermines that it is not the last divisional translation text in a casein which there is an unprocessed translation result, and determines thatit is the last divisional translation text in a case in which there isno unprocessed translation result.

In a case in which it is determined in step S1206 that it is not thelast divisional translation text, the information processing apparatusaccording to the present embodiment causes a sound “beep” to be outputas sound feedback for showing that it continues even afterward (S1208).Then, the information processing apparatus according to the presentembodiment repeats the process starting from step S1200.

Further, in a case in which it is determined in step S1206 that it isthe last divisional translation text, the information processingapparatus according to the present embodiment causes a sound “beep beep”to be output as sound feedback for showing that it is the last (S1210).Then, the information processing apparatus according to the presentembodiment repeats the process starting from step S1200.

In a case in which it is determined in step S1200 that there is nodivisional translation text to be processed, the information processingapparatus according to the present embodiment ends the process of FIG.33.

The information processing apparatus according to the present embodimentperforms, for example, the process illustrated in FIG. 33 as the processof step S916 in FIG. 30. Further, needless to say, the process of stepS916 in FIG. 30 is not limited to the process illustrated in FIG. 33.

For example, as the processes illustrated in FIGS. 22 to 33 areperformed, the use case described with reference to FIGS. 1, 2, 3A, 3B,3C, 4A, 4B, and 5 can be implemented. Further, needless to say, theprocess according to the information processing method of the presentembodiment is not limited to the processes illustrated in FIGS. 22 to33.

[5] Example of Effect Obtained by Using Information Processing MethodAccording to Present Embodiment

The information processing apparatus according to the present embodimentperforms the process according to the information processing method ofthe present embodiment, and thus for example, the following effects areobtained. Further, needless to say, the effects obtained by using theinformation processing method of the present embodiment are not limitedto the following effects.

-   -   Even in a case in which the speaker speaks in a disorganized        manner, only the main points are translated, and thus matters        which the speaker desires to convey can be conveyed to a        recipient.    -   Since only the main points are translated, it is possible to        shorten a confirmation time of the recipient and realize smooth        translation communication.    -   Since there are cases in which the sentences serving as the        processing target of the translation process can be reduced        extremely, the accuracy of translation can be improved.    -   Since the recipient need not receive unnecessary words since the        content of the speech is summarized and then translated, it is        easy for the recipient to understand. As a result, it is        possible to encourage people who are not good at foreign        languages to speak across language barriers.        (Information Processing Apparatus According to Embodiment)

Next, an example of a configuration of the information processingapparatus according to the embodiment capable of performing the processrelated to the information processing method according to the embodimentdescribed above will be described. Hereinafter, as one example of aconfiguration of the information processing apparatus according to thepresent embodiment, example of an information processing apparatuscapable of performing either or both of the process according to thefirst information processing method described above and the processaccording to the second information processing method described abovewill be described.

FIG. 34 is a block diagram illustrating an example of a configuration ofthe information processing apparatus 100 according to the embodiment.The information processing apparatus 100 includes, for example, acommunication unit 102 and a control unit 104.

Moreover, for example, the information processing apparatus 100 mayinclude, a read-only memory (ROM which is not illustrated), a randomaccess memory (RAM which is not illustrated), a storage unit (notillustrated), a manipulation unit (not illustrated) which can bemanipulated by a user of the information processing apparatus 100, and adisplay unit (not illustrated) that displays various screens on adisplay screen. In the information processing apparatus 100, forexample, the constituent elements are connected via a bus serving as adata transmission path. Further, the information processing apparatus100 is driven by, for example, electric power supplied from an internalpower supply such as a battery of the information processing apparatus100, electric power supplied from a connected external power supply, orthe like.

The ROM (not illustrated) stores a program or control data such ascalculation parameters which are used by the control unit 104. The RAM(not illustrated) temporarily stores a program or the like which isexecuted by the control unit 104.

The storage unit (not illustrated) is storage means included in theinformation processing apparatus 100 and stores, for example, datarelated to the information processing method according to theembodiment, such as the table for setting the weight related to thesummary and various kinds of data such as various applications. Here, asthe storage unit (not illustrated), for example, a magnetic recordingmedium such as a hard disk and a nonvolatile memory such as a flashmemory can be exemplified. Moreover, the storage unit (not illustrated)may be detachably mounted on the information processing apparatus 100.

As the manipulation unit (not illustrated), a manipulation input deviceto be described below can be exemplified. Moreover, as the display unit(not illustrated), a display device to be described below can beexemplified.

[Example of Hardware Configuration of Information Processing Apparatus100]

FIG. 35 is an explanatory diagram illustrating an example of a hardwareconfiguration of the information processing apparatus 100 according tothe embodiment. The information processing apparatus 100 includes, forexample, an MPU 150, a ROM 152, a RAM 154, a recording medium 156, aninput and output interface 158, a manipulation input device 160, adisplay device 162, and a communication interface 164. Moreover, in theinformation processing apparatus 100, for example, the constituentelements are connected via a bus 166 serving as a data transmissionpath.

For example, the MPU 150 includes one or two or more processors orvarious processing circuits including a calculation circuit such as anMPU, and functions as the control unit 104 controlling the entireinformation processing apparatus 100. Moreover, the MPU 150 plays rolesof, for example, the processing unit 110 to be described below in theinformation processing apparatus 100. Note that the processing unit 110may include a dedicated (or general-purpose) circuit (for example, aprocessor different from the MPU 150) capable of realizing a process ofthe processing unit 110.

The ROM 152 stores a program or control data such as calculationparameters which is used by the MPU 150. For example, the RAM 154temporarily stores a program or the like which is executed by the MPU150.

The recording medium 156 functions as a storage unit (not illustrated)and stores, for example, data related to the information processingmethod according to the embodiment, such as the table for setting theweight related to the summary and various kinds of data such as variousapplications. Here, as the recording medium 156, for example, a magneticrecording medium such as a hard disk and a nonvolatile memory such as aflash memory can be exemplified. Moreover, the recording medium 156 maybe detachably mounted on the information processing apparatus 100.

The input and output interface 158 connects, for example, themanipulation input device 160 or the display device 162. Themanipulation input device 160 functions as a manipulation unit (notillustrated) and the display device 162 functions as a display unit (notillustrated). Here, as the input and output interface 158, for example,a Universal Serial Bus (USB) terminal, a Digital Visual Interface (DVI)terminal, a High-Definition Multimedia Interface (HDMI) (registeredtrademark) terminal, and various processing circuits can be exemplified.

Moreover, for example, the manipulation input device 160 can beinstalled on the information processing apparatus 100 and is connectedto the input and output interface 158 inside the information processingapparatus 100. As the manipulation input device 160, for example, abutton, an arrow key, a rotary selector such as a jog dial, or acombination thereof can be exemplified.

Moreover, for example, the display device 162 can be installed on theinformation processing apparatus 100 and is connected to the input andoutput interface 158 inside the information processing apparatus 100. Asthe display device 162, for example, a liquid crystal display or anorganic electro-luminescence display (or also referred to as an organiclight emitting diode display) can be exemplified.

In addition, it is regardless to say that the input and output interface158 can be connected to an external device such as an externalmanipulation input device (for example, a keyboard or a mouse) or anexternal display device of the information processing apparatus 100.Moreover, the display device 162 may be, for example, a device in whichdisplay and a user manipulation are possible, such as a touch panel.

The communication interface 164 is communication means included in theinformation processing apparatus 100 and functions as the communicationunit 102 that performs wireless or wired communication with an externalapparatus or an external device. Here, as the communication interface164, for example, a communication antenna and a radio frequency (RF)circuit (for wireless communication), an IEEE 802.15.1 port and atransmission and reception circuit (for wireless communication), an IEEE802.11 port and a transmission and reception circuit (for wirelesscommunication), or a Local Area Network (LAN) terminal and atransmission and reception circuit (for wired communication) can beexemplified.

The information processing apparatus 100 that has, for example, theconfiguration illustrated in FIG. 35 performs the process related to theinformation processing method according to the embodiment. In addition,a hardware configuration of the information processing apparatus 100according to the embodiment is not limited to the configurationillustrated in FIG. 35.

For example, the information processing apparatus 100 does not have toinclude the communication interface 164 in a case in which communicationwith an external apparatus is performed via a connected externalcommunication device. Moreover, the communication interface 164 may becapable of performing communication with one or two or more externalapparatuses in conformity with a plurality of communication schemes.

Moreover, for example, the information processing apparatus 100 does nothave to include the recording medium 156, the manipulation input device160, or the display device 162.

Further, the information processing apparatus 100 may further includeone or more of, for example, various sensors such as a motion sensor anda biosensor, a voice input device such as a microphone, a voice outputdevice such as a speaker, a vibration device, an imaging device, and thelike.

Moreover, a part or the whole of the structural elements illustrated inFIG. 35 (or a configuration according to a modification example) may berealized by one or two or more ICs.

Referring back to FIG. 34, an example of the configuration of theinformation processing apparatus 100 will be described. Thecommunication unit 102 is communication means included in theinformation processing apparatus 100 and performs wireless or wiredcommunication with an external apparatus or an external device via anetwork (or directly). Moreover, the communication of the communicationunit 102 is controlled by, for example, the control unit 104.

Here, as the communication unit 102, for example, a communicationantenna and an RF circuit or a LAN terminal and a transmission andreception circuit can be exemplified. However, the configuration of thecommunication unit 102 is not limited to the foregoing configuration.For example, the communication unit 102 can have a configurationcorresponding to any standard capable of performing communication, suchas a USB terminal and a transmission and reception circuit or have anyconfiguration capable of communicating an external apparatus via anetwork. Moreover, the communication unit 102 may have a configurationcapable of performing communication with one or two or more externalapparatuses in conformity with a plurality of communication schemes.

The control unit 104 is constituted by, for example, an MPU or the likeand plays a role of controlling the information processing apparatus 100in general. Further, the control unit 104 includes, for example, aprocessing unit 110, and plays a leading role of performing the processaccording to the information processing method of the presentembodiment. For example, the processing unit 110 plays a leading role ofperforming either or both of the process according to the firstinformation processing method described above and the process accordingto the second information processing method described above.

In a case in which the process according to the first informationprocessing method described above is performed, the processing unit 110performs the summarization process of summarizing the content of thespeech indicated by the voice information on the basis of the acquiredinformation indicating the weight related to the summary. The processingunit 110 performs, for example, the process described in section [3-1]as the summarization process.

In a case in which the processing according to the second informationprocessing method described above is performed, the processing unit 110performs the notification control process for controlling thenotification of the notification content on the basis of the summaryinformation. The processing unit 110 performs, for example, the processdescribed in section [3-3] as the notification control process.

Further, the processing unit 110 may further perform the translationprocess of translating the content of the speech summarized according tothe summarization process into another language. The processing unit 110performs, for example, the process described in section [3-2] as thetranslation process.

In a case in which the summarized content of the speech is translatedinto another language according to the translation process, theprocessing unit 110 can cause a notification of the translation resultto be given according to the notification control process.

Further, the processing unit 110 can perform various kinds of processesaccording to the information processing method of the present embodimentsuch as the process related to the voice recognition, the processrelated to the voice analysis, the process related to the estimation ofthe state of the user, and the process related to the estimation of thedistance between the user and the communication partner. Further,various kinds of processes according to the information processingmethod of the present embodiment may be performed in the externalapparatus of the information processing apparatus 100.

For example, with the configuration illustrated in FIG. 34, theinformation processing apparatus 100 performs the process according tothe information processing method of the present embodiment (forexample, “either or both of the summarization process according to thefirst information processing method and the notification control processaccording to the second information processing method” or “either orboth of the summarization process according to the first informationprocessing method and the notification control process according to thesecond information processing method and the translation process”).

Therefore, in a case in which the summarization process according to thefirst information processing method is performed as the processaccording to the information processing method of the presentembodiment, the information processing apparatus 100 can summarize thecontent of the speech, for example, according to the configurationillustrated in FIG. 34.

Further, in a case in which the notification control process accordingto the second information processing method is performed as the processaccording to the information processing method of the presentembodiment, the information processing apparatus 100 can cause anotification of the summarized content of the speech to be given, forexample, according to the configuration illustrated in FIG. 34.

Further, as the information processing apparatus 100 can obtain theeffects obtained by performing the process according to the informationprocessing method of the present embodiment as described above, forexample, with the configuration illustrated in FIG. 34.

Further, the configuration of the information processing apparatusaccording to the present embodiment is not limited to the configurationillustrated in FIG. 34.

For example, in the information processing apparatus according to thepresent embodiment, the processing unit 110 illustrated in FIG. 34 canbe installed separately from the control unit 104 (for example, can beimplemented by another processing circuit). Further, for example, thesummarization process according to the first information processingmethod, the notification control process according to the secondinformation processing method, and the translation process according tothe present embodiment may be distributed and performed through aplurality of processing circuits.

Further, the summarization process according to the first informationprocessing method, the notification control process according to thesecond information processing method, and the translation processaccording to the present embodiment are processes in which the processaccording to the information processing method of the present embodimentis specified for the sake of convenience. Therefore, the configurationfor implementing the process according to the information processingmethod of the present embodiment is not limited to the configurationillustrated in FIG. 34, and a configuration corresponding to a way ofseparating the process according to the information processing method ofthe present embodiment may be employed.

Further, for example, in a case in which communication is performed withan external apparatus through an external communication device having asimilar function and configuration to those of the communication unit102, the information processing apparatus according to the presentembodiment may not include the communication unit 102.

The present embodiment has been described using an informationprocessing apparatus as an example, but the present embodiment is notlimited to such an example. The present embodiment can be applied tovarious devices which are capable of performing the processes accordingto the information processing method of the present embodiment (forexample, either of both of the process according to the firstinformation processing method and the process according to the secondinformation processing method) such as a “computer” such as a personalcomputer (PC) or a server,” an “arbitrary wearable apparatus which isworn on the body of the user and used such as an eyewear type apparatus,a clock type apparatus, or a bracelet type apparatus,” a “communicationapparatus such as a smartphone,” a “tablet type apparatus,” a “gamemachine,” and a “mobile object such as automobile.” Further, forexample, the present embodiment can also be applied to a processing ICwhich can be incorporated in devices described above.

Further, the information processing apparatus according to the presentembodiment may be applied to a processing system based on a connectionto a network (or communication between respective apparatuses) such ascloud computing or the like. As an example of the processing system inwhich the process according to the information processing method of thepresent embodiment is performed, for example, a “system in which thesummarization process according to the first information processingmethod and the translation process is performed by one apparatusconstituting the processing system, and the notification control processaccording to the second information processing method is performed byanother apparatus constituting the processing system” may be used.

(Program According to Present Embodiment)

[I] Program (Computer Program) According to First Information ProcessingMethod

A program causing a computer to function as the information processingapparatus according to the present embodiment performing the processaccording to the first information processing method (for example, aprogram capable of executing the process according to the firstinformation processing method such as “the summarization processaccording to the first information processing method” or “thesummarization process according to the first information processingmethod and the translation process according to the present embodiment”)is executed by a processor or the like in a computer, and thus contentof speech can be summarized.

Further, the program causing the computer to function as the informationprocessing apparatus according to the present embodiment performing theprocess according to the first information processing method is executedby the processor or the like in the computer, and thus the effectsobtained by the process according to the first information processingmethod described above can be obtained.

[II] Program According to Second Information Processing Method

A program causing a computer to function as the information processingapparatus according to the present embodiment performing the processaccording to the second information processing method (for example, aprogram capable of executing the process according to the secondinformation processing method such as “the notification control processaccording to the second information processing method” or “thetranslation process according to the present embodiment and thenotification control process according to the second informationprocessing method”) is executed by a processor or the like in acomputer, and thus it is possible to cause a notification of content ofspeech to be given.

Further, the program causing the computer to function as the informationprocessing apparatus according to the present embodiment performing theprocess according to the second information processing method isexecuted by the processor or the like in the computer, and thus theeffects obtained by the process according to the second informationprocessing method described above can be obtained.

[III] Program According to Information Processing Method of PresentEmbodiment

The program related to the information processing method of the presentembodiment may include both a program according to the first informationprocessing method and a program according to the second informationprocessing method.

The preferred embodiment(s) of the present disclosure has/have beendescribed above with reference to the accompanying drawings, whilst thepresent disclosure is not limited to the above examples. A personskilled in the art may find various alterations and modifications withinthe scope of the appended claims, and it should be understood that theywill naturally come under the technical scope of the present disclosure.

For example, the example in which the program causing the computer tofunction as the information processing apparatus according to thepresent embodiment (the program capable of executing either or both ofthe process according to the first information processing method and theprocess according to the second information processing method) isprovided has been described above, but in the present embodiment, arecording medium having the program stored therein can be providedtogether.

The above-described configuration is a configuration indicating anexample of the present embodiment, and it would be understood that theabove-described configuration is included in the technical scope of thepresent disclosure.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing apparatus including:

a processing unit configured to perform a summarization process ofsummarizing content of speech indicated by voice information based onspeech of a user on a basis of acquired information indicating a weightrelated to a summary.

(2)

The information processing apparatus according to (1), in which theprocessing unit performs the summarization process in a case in which apredetermined start condition is determined to be satisfied.

(3)

The information processing apparatus according to (2), in which thestart condition is a condition related to a non-speaking period in whicha state in which speech is not performed continues, and

in a case in which the non-speaking period exceeds a predeterminedperiod or in a case in which the non-speaking period is equal to orlarger than the predetermined period, the processing unit determinesthat the start condition is satisfied.

(4)

The information processing apparatus according to (2) or (3), in whichthe start condition is a condition related to a state of voicerecognition for acquiring the content of the speech from the voiceinformation, and

the processing unit determines that the start condition is satisfied ona basis of detection of a stop request for the voice recognition.

(5)

The information processing apparatus according to any one of (2) to (4),in which the start condition is a condition related to a state of voicerecognition for acquiring the content of the speech from the voiceinformation, and

the processing unit determines that the start condition is satisfied ona basis of detection of completion of the voice recognition.

(6)

The information processing apparatus according to any one of (2) to (5),in which the start condition is a condition related to the content ofthe speech, and

the processing unit determines that the start condition is satisfied ona basis of detection of a predetermined word from the content of thespeech indicated by the voice information.

(7)

The information processing apparatus according to any one of (2) to (6),in which the start condition is a condition related to the content ofthe speech, and

the processing unit determines that the start condition is satisfied ona basis of detection of hesitation to speak, the detection being basedon the voice information.

(8)

The information processing apparatus according to any one of (2) to (7),in which the start condition is a condition related to an elapsed timesince the voice information is obtained, and

the processing unit determines that the start condition is satisfied ina case in which the elapsed time exceeds a predetermined period or in acase in which the elapsed time is equal to or larger than thepredetermined period.

(9)

The information processing apparatus according to any one of (1) to (8),in which the processing unit does not perform the summarization processin a case in which a predetermined summarization exclusion condition isdetermined to be satisfied.

(10)

The information processing apparatus according to (9), in which thesummarization exclusion condition is a condition related to detection ofa gesture, and

the processing unit determines that the summarization exclusioncondition is satisfied in a case in which a predetermined gesture isdetected.

(11)

The information processing apparatus according to any one of (1) to(10), in which the processing unit changes a summarization level of thecontent of the speech on a basis of at least one of a speaking periodspecified on a basis of the voice information and a number of charactersspecified on the basis of the voice information.

(12)

The information processing apparatus according to (11), in which theprocessing unit changes the summarization level of the content of thespeech by limiting the number of characters indicated by the summarizedcontent of the speech.

(13)

The information processing apparatus according to any one of (1) to(12), in which the processing unit sets the weight related to thesummary on a basis of at least one of the voice information, informationrelated to the user, information related to an application, informationrelated to an environment, and information related to a device.

(14)

The information processing apparatus according to (13), in which theinformation related to the user includes at least one of stateinformation of the user and manipulation information of the user.

(15)

The information processing apparatus according to any one of (1) to(14), in which the processing unit further performs a translationprocess of translating the content of the speech summarized according tothe summarization process into another language.

(16)

The information processing apparatus according to (15), in which theprocessing unit does not perform the translation process in a case inwhich a predetermined translation exclusion condition is determined tobe satisfied.

(17)

The information processing apparatus according to (15) or (16), in whichthe processing unit retranslates the content translated into anotherlanguage according to the translation process into a language beforetranslation, and

in a case in which a word included in the content after retranslation ispresent in the content of the speech indicated by the voice informationacquired after the retranslation, the word included in the content afterthe retranslation is included in the summarized content of the speech.

(18)

The information processing apparatus according to any one of (1) to(17), in which the processing unit further performs a notificationcontrol process of controlling notification of the summarized content ofthe speech.

(19)

An information processing method that is executed by an informationprocessing apparatus, the information processing method including:

a step of performing a summarization process of summarizing content ofspeech indicated by voice information based on speech of a user on abasis of acquired information indicating a weight related to a summary.

(20)

A program for causing a computer to implement:

a function of performing a summarization process of summarizing contentof speech indicated by voice information based on speech of a user on abasis of acquired information indicating a weight related to a summary.

REFERENCE SIGNS LIST

-   100 information processing apparatus-   102 communication unit-   104 control unit-   110 processing unit

The invention claimed is:
 1. An information processing apparatus,comprising: at least one processor configured to: detect a first speechof a user based on first voice information associated with the firstspeech of the user; set a weight associated with a summary of the firstspeech of the user, wherein the weight is set based on informationrelated to an application for execution by the user; summarize firstcontent of the first speech indicated by the first voice information togenerate the summary of the first speech, wherein the summarization ofthe first content is based on the set weight associated with thesummary; detect a first language of the first speech of the user;translate the first content of the first speech into a second languagevia a translation process, wherein the second language is different fromthe first language; retranslate the first content, which is translatedinto the second language via the translation process, into the firstlanguage; acquire second voice information, after the retranslation ofthe first content; and include a first word in the first content of thefirst speech associated with the first voice information, based on thefirst word which is present in each of the retranslated first contentand second content of a second speech associated with the second voiceinformation.
 2. The information processing apparatus according to claim1, wherein the summarization of the first content is further based on astart condition being satisfied.
 3. The information processing apparatusaccording to claim 2, wherein the start condition is related to anon-speaking period in which the first speech is not performed by theuser, and the at least one processor is further configured to determinethat the start condition is satisfied based on the non-speaking periodwhich is one of equal to or larger than a specific period.
 4. Theinformation processing apparatus according to claim 2, wherein the startcondition is related to a state of a voice recognition operation toacquire the first content of the first speech from the first voiceinformation, and the at least one processor is further configured todetermine that the start condition is satisfied based on a stop requestfor the voice recognition operation.
 5. The information processingapparatus according to claim 2, wherein the start condition is relatedto a state of a voice recognition operation to acquire the first contentof the first speech from the first voice information, and the at leastone processor is further configured to determine that the startcondition is satisfied based on completion of the voice recognitionoperation.
 6. The information processing apparatus according to claim 2,wherein the start condition is related to the first content of the firstspeech, and the at least one processor is further configured todetermine that the start condition is satisfied based on a second wordin the first content of the first speech indicated by the first voiceinformation.
 7. The information processing apparatus according to claim2, wherein the start condition is related to the first content of thefirst speech, and the at least one processor is further configured to:detect a hesitation of the user to speak, based on the first voiceinformation; and determine that the start condition is satisfied basedon the detection of the hesitation to speak.
 8. The informationprocessing apparatus according to claim 2, wherein the start conditionis related to an elapsed time since the first voice information isobtained, and the at least one processor is further configured todetermine that the start condition is satisfied based on the elapsedtime which is one of equal to or larger than a specific period.
 9. Theinformation processing apparatus according to claim 1, wherein the atleast one processor is further configured to abort the summarization,based on a summarization exclusion condition is satisfied.
 10. Theinformation processing apparatus according to claim 9, wherein thesummarization exclusion condition is related to detection of a gesture,and the at least one processor is further configured to determine thatthe summarization exclusion condition is satisfied based on thedetection of the gesture.
 11. The information processing apparatusaccording to claim 1, wherein the at least one processor is furtherconfigured to change a summarization level of the first content of thefirst speech based on at least one of a speaking period specified by thefirst voice information and a number of characters specified by thefirst voice information.
 12. The information processing apparatusaccording to claim 11, wherein the at least one processor is furtherconfigured to limit the number of characters in the summary of the firstspeech to change the summarization level of the first content of thefirst speech.
 13. The information processing apparatus according toclaim 1, wherein the at least one processor is further configured to setthe weight associated with the summary, based on at least one ofinformation related to the user, information related to an environmentassociated with the user, or information related to a device, and theinformation related to the device includes at least one of a type of thedevice or a state of the device.
 14. The information processingapparatus according to claim 13, wherein the information related to theuser includes at least one of state information of the user ormanipulation information of the user.
 15. The information processingapparatus according to claim 1, wherein the at least one processor isfurther configured to abort the translation based on a translationexclusion condition being satisfied.
 16. The information processingapparatus according to claim 1, wherein the at least one processor isfurther configured to control notification of the first content of thefirst speech.
 17. An information processing method, comprising:detecting a first speech of a user based on first voice informationassociated with the first speech of the user; setting a weightassociated with a summary of the first speech of the user, wherein theweight is set based on information related to an application forexecution by the user; summarizing first content of the first speechindicated by the first voice information to generate the summary of thefirst speech, wherein the summarization of the first content is based onthe set weight associated with the summary; detecting a first languageof the first speech of the user; translating the first content of thefirst speech into a second language via a translation process, whereinthe second language is different from the first language; retranslatingthe first content, which is translated into the second language via thetranslation process, into the first language; acquiring second voiceinformation, after the retranslation of the first content; and includinga word in the first content of the first speech associated with thefirst voice information, based on the word which is present in each ofthe retranslated first content and second content of a second speechassociated with the second voice information.
 18. A non-transitorycomputer-readable medium having stored thereon, computer-executableinstructions which, when executed by a computer, cause the computer toexecute operations, the operations comprising: detecting a first speechof a user based on first voice information associated with the firstspeech of the user; setting a weight associated with a summary of thefirst speech of the user, wherein the weight is set based on informationrelated to an application for execution by the user; summarizing firstcontent of the first speech indicated by the first voice information forgeneration of the summary of the first speech, wherein the summarizationof the first content is based on the set weight associated with thesummary; detecting a first language of the first speech of the user;translating the first content of the first speech into a second languagevia a translation process, wherein the second language is different fromthe first language; retranslating the first content, which is translatedinto the second language via the translation process, into the firstlanguage; acquiring second voice information, after the retranslation ofthe first content; and including a word in the first content of thefirst speech associated with the first voice information, based on theword which is present in each of the retranslated first content andsecond content of a second speech associated with the second voiceinformation.